Psst....Type Hinting 🤫
Part 1 of 2
Everything you read starts with one question - Why do I care?
Today I’ll tell you why you should (well, maybe not Should - but it would be nice if you did) care about Type Hinting. This is Part 1 of a 2-part series. The whole post might not be rendered in your email. Please check out the original post on Substack for the full article.
🧪 For this post, I worked with Python3.4 and Python3.10 - the latest version. All the code snippets below and the outputs pasted are from execution in Python 3.10. Most of the syntaxes used in the code snippets should however be supported by all versions >= 3.5.
I learnt about Type Hinting in Python very recently, in retrospect, because while we are on Python3.10 now, Type Hinting was formally incorporated into Python in version 3.5. Now types and static typing is a much larger subject. This article and the next are limited to discussing PEP 484, the proposal that along with PEP 483, announced the concept, and how type hinting can be used in your code. This should suffice for bettering your code immediately but I’d definitely encourage more study for the interested (links at end of post).
Python is a statically typed language, unlike C++ or Java. Which means, first of all, there’s no concept of “declaring” a variable like `int a = 10;` and secondly, when you use parameters in a function signature, you don’t specify its type there either. There is no compile-time checking to see if a variable used is of a type as intended by the user. It’s only at runtime, based on operations possible between variables of different types that it’s determined if everything is in place or not.
So, this degree-of-freedom offered to programmers is a good thing right? Perhaps.
But where the flexibility of dynamically-typed Python makes writing code easier, it also makes it easy for type-based bugs that could have been caught sooner than in Production to creep in. Enter “optional static typing”.
Optional Static Typing was originally a notion proposed by Guido Van Rossum in 2004. But it took 11 more years for the language to adopt it officially, making its way from Mypy.
The road to this was slow and deliberate. Mypy was originally started by Jukka in 2012. In the beginning, Mypy programs that supported some type hinting, required a translation step before being supported by Python interpreter. It was only after PyCon in 2013, that Jukka started working on making Mypy programs compatible with CPython.
PEP 484 - Introduction
What should be noted at the outset is that, type-hinting is not Enforced by the Python interpreter. This task of checking is delegated to third-party libraries like mypy, pyre, pyright etc. Here, I will be working with mypy, unless mentioned otherwise.
🧪 The way to use mypy as a typechecker is to install it using pip and passing the name of the script as an argument to mypy. Example -
mypy script1.py a) If a function or a block of code is dynamically typed, the type-checker will ignore it.
This function is how you probably are used to writing code. No typing-specific annotations or type-hints introduced yet. So when you run it through mypy, it does nothing.
b) How else do you make your type-checker ignore a function?
For now, focus only on the first line in the snippet above. @no_type_check - That’s the annotation (to be imported from the typing module) that exempts the entire function from the type-checker’s purview.
c) So what’s a legit type-hint?
Consider the above function. The signature of the function (within parentheses) is different from what you’re used to and there’s something separating the arguments from the “:” that signifies the beginning of the function. These are called type-hints. In this example, we’re saying there’s a parameter “li” that’s expected to be a list of ints.
The return type of the function is expected to be a tuple consisting of a string and an integer in that order. While list, int and str are primitive datatypes, Tuple is imported from the typing module.
This may seem like a chore to begin with. And for such a simple module, it probably is. But as your codebase expands and you have hundreds of functions across tens of files as you probably would in a production setup, it’s handy to look at a function and know immediately what type of parameters it expects and what it’s expected to return.
Now, how do we check these typehints being enforced?
Hello mypy!
Now consider the 3 functions from (a), (b) and (c) being called in sequence. They literally perform the same function, and are theoretically expected to return the same result. And they do, Kevin -
And if we use the mypy typechecker on the entire script (which can be found on the Github repo for this post) like so -
Heh heh. No issues found.
But, if you look at (b) again, the function we sneakily made the type-checker ignore, you’ll notice that the return type expected is Tuple[int, int] but we’re returning a string and an integer. So, for the purposes of this test, I’ll rerun the typechecker having removed the annotation -
You’re BUSTED, Uncle Leo 😏
🧪 Other usable type Constructs in addition to Tuple that can be used are - None, Any, Union, Callable, all Abstract Base Classes and concrete classes exported from typing (e.g. Sequence and Dict), type variables, and type aliases.
Type Aliases & Callable
Now, as already mentioned earlier, one of the reasons typehints are useful is to improve readability of code. To that end, as important as variable names are in indicating what the variables actually mean contextually to the program, it’d would be cool to have datatypes add an additional layer of meaning. They can be used to alias primitive types and User-defined types as well.
Consider the following snippet -
Now, this function is simply computing the day of week for a given date string. But one of the most confusing inputs is the format of date expected by the program. As far as the user input is concerned, that can be handled by providing a helpful note while accepting said input. But the function’s readability can also be improved by aliasing the expected date’s type (here, a string) to something like “YYYY_MM_DD” and it can be understood that dt is expected to be of this format.
🧪 The above example is indicative. Ideally the format expected can also be included in the function’s docstring. There are probably better uses for type aliasing.
Some functions might expected a callable function parameter and the type hints for these can be represented using Callable.
Syntax : Callable[[Arg1Type, Arg2Type], ReturnType]I cannot think of an example for this at the moment. I’ll edit this post later when I think of one.
Generics
Consider containers (like tuple, list, dict, set). Now the type of objects within these containers cannot be naively inferred. This is why Generics are useful. Their usage is facilitated by a Factory in the typing method called TypeVar. The syntax for its declaration is -
generic_variable = TypeVar(generic_variable)But mypy requires us to constrain the type of the Generic further. For example,
In this example, we have 3 Generic types - T, T2 and T3, of which T is a vanilla generic and T2 and T3 are constrained to the class type ‘Rectangle’ or str or float or the typing construct ‘Any’.
All 4 functions again perform exactly the same…well..function 😸
But because of their corresponding typehints, mypy respects them differently -
Constraining a Generic is as follows -
gen_var = TypeVar(gen_var, type1, type2, type3,…typeN)The reason I repeated gen_var both on the LHS and within the argument list is because that’s what TypeVar expects - that the first parameter be the same as the Generic type’s name. The rest of the parameters are types that can be used for any typehint - primitive, user-defined etc.
In the example above, T2 can either be of type Rectangle or Any and T3 is expected to be of str or float type.
In the mypy errors displayed -
a) T has no type defined per se. Which is why errors 1 and 2 can’t understand what “length” and “breadth” mean to it.
b) Errors 3 and 4 are because of T3 which expects a str or float, but the method “area3” uses Rectangle’s attributes while the typehint for its argument is set to T3. This can mean either a mismatched Typehint Or an incorrect attribute was passed. Here, the former applies.
c) Errors 5 and 6 are because of (b) as well. Except these are indicated at the calling statements (can be looked up on the GitHub repo) .
This marks the end of this part of Type-Hinting as far as PEP 484 is concerned. The rest of it will be continued in Part 2 !
While this is probably not the most interesting of topics as it gets deeper into the woods, but there are a lot of underlying concepts I’ve gotten to understand in writing this.
🧪 Apart from mypy, other typecheckers are pyright, pyre and pytype. I tried pyright and pytype. pyright seems to have a lot of gaps in implementation and pytype does not support Python3.10 yet, so we’ll continue using mypy. Pycharm, my IDE of usage here also has a built in typechecker which is pretty awesome.
Resources and code -
The code accompanying this post can be found at - https://github.com/everythingpython/post2