Do you know the difference between the following syntax?<\/p>\n
[x for x in range(5)]<\/strong> This is exactly what differentiates Python from other languages and makes Python the best programming language<\/a>. Coming from functional languages and being implemented in Python from early days, list comprehension became its distinctive feature.<\/p>\n This article is a complete guide to list comprehensions and generator expressions in Python. It provides many practical examples of their usage.<\/p>\n You\u2019ll know:<\/p>\n Let\u2019s dive deeper into the Python list comprehensions and generator expressions. In Django Stars, a Python development company<\/a>, we often use list comprehensions in our projects<\/a>, so if you need additional advice on Python or Django development<\/a>, you just need to contact us<\/a>.<\/p>\n Read Also: Debugging Python Applications with the PDB Module<\/a><\/p>\n First off, a short review on the lists (arrays in other languages).<\/p>\n There are 2 common ways how to create lists in Python:<\/p>\n And less preferable:<\/p>\n Usually, list(obj)<\/b> is used to transform another sequence into the list. For example we want to split string into separate symbols:<\/p>\n Often seen as a part of functional programming in Python, list comprehension allows you to create lists with less code. List comprehensions enhance Python code performance<\/a> by being faster and more efficient than traditional Let\u2019s look at the following example.<\/p>\n You create a list using a for loop and a range()<\/strong> function.<\/p>\n And this is how the implementation of the previous example is performed using a list comprehension:<\/p>\n The above example is oversimplified to get the idea of syntax. The same result may be achieved simply using list(range(0, 19, 2))<\/strong> function. However, you can use a more complex modifier in the first part of the comprehension or add a condition that will filter the list. Something like this:<\/p>\n Another available option is to use list comprehension to combine several lists and create a list of lists. At first glance, the syntax seems to be complicated. It may help to think of lists as outer and inner sequences. Let\u2019s try it with text or, referring to it correctly, string object.<\/p>\n The comprehensions are not limited to lists. You can create dicts and sets comprehensions as well with the Python set generator.<\/p>\n List comprehension is the best way to enhance the code readability, so it’s worth using whenever there is a bunch of data to be checked with the same function or logic \u2013 for example, KYC verification. If the logic is quite simple, for instance, it’s limited with `true` or `false` results, list comprehension can optimize the code and focus on the logic solely. For example:<\/p>\n There are many other ways how it can be implemented, but let’s have a look at the example with list comprehension:<\/p>\n List comprehensions optimize the lists\u2019 generation and help to avoid side effects as gibberish variables. As a result, you get more concise and readable code. It will be easier to understand the concept of Python list generators if you get the idea of iterables and iterators. Iterator protocol is implemented whenever you iterate over a sequence of data. For example, when you use a for loop the following is happening on a background:<\/p>\n In Python, generators provide a convenient way to implement the iterator protocol. Generator is an iterable created using a function with a yield<\/b> statement. A Python generator expression is an expression that returns a generator (generator object). In terms of syntax, the only difference is that you use parentheses instead of square brackets. However, the types of data returned by Python generator expressions and list comprehensions differ.<\/p>\n The main advantage of generator over a list is that it takes much less memory. We can check how much memory is taken by both types using sys.getsizeof()<\/strong> method. We can see this difference because while `list` creating Python reserves memory for the whole list and calculates it on the spot. In case of generator, we receive only ”algorithm”\/ “instructions” how to calculate that Python stores. And each time we call for generator, it will only \u201cgenerate\u201d the next element of the sequence on demand according to “instructions”.
\n(x for x in range(5))<\/strong>
\ntuple(range(5))<\/strong><\/p>\n\n
4 Facts About the Lists<\/h2>\n
\n
>>> a = 12\r\n>>> b = \"this is text\"\r\n>>> my_list = [0, b, ['element', 'another element'], (1, 2, 3), a]\r\n>>> print(my_list)\r\n[0, 'this is text', ['element', 'another element'], (1, 2, 3), 12]\r\n<\/code><\/pre>\n\n
>>> a = ['red', 'green', 'blue']\r\n>>> print(a[0])\r\nred\r\n<\/code><\/pre>\n\n
How to Create a List in Python<\/h3>\n
>>> my_list = [0, 1, 1, 2, 3]<\/code><\/pre>\n>>> my_list = list()<\/code><\/pre>\n>>> string = \"string\"\r\n>>> list(string)\r\n['s', 't', 'r', 'i', 'n', 'g']<\/code><\/pre>\nWhat is List Comprehension?<\/h2>\n
for<\/code> loops. They also improve code readability and conciseness, making it easier to write and maintain. In short, it\u2019s a truly Pythonic way of coding. Less code – more effectiveness.<\/p>\n>>> my_list = []\r\n>>> for x in range(10):\r\n... my_list.append(x * 2)\r\n...\r\n>>> print(my_list)\r\n[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]\r\n<\/code><\/pre>\n>>> comp_list = [x * 2 for x in range(10)]\r\n>>> print(comp_list)\r\n[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]\r\n<\/code><\/pre>\n>>> comp_list = [x ** 2 for x in range(7) if x % 2 == 0]\r\n>>> print(comp_list)\r\n[4, 16, 36]\r\n<\/code><\/pre>\n
\nIt\u2019s time to show the power of list comprehension when you want to create a list of lists by combining two existing lists.<\/p>\n>>> nums = [1, 2, 3, 4, 5]\r\n>>> letters = ['A', 'B', 'C', 'D', 'E']\r\n>>> nums_letters = [[n, l] for n in nums for l in letters]\r\n#the comprehensions list combines two simple lists in a complex list of lists.\r\n>>> print(nums_letters)\r\n>>> print(nums_letters)\r\n[[1, 'A'], [1, 'B'], [1, 'C'], [1, 'D'], [1, 'E'], [2, 'A'], [2, 'B'], [2, 'C'], [2, 'D'], [2, 'E'], [3, 'A'], [3, 'B'], [3, 'C'], [3, 'D'], [3, 'E'], [4, 'A'], [4, 'B'], [4, 'C'], [4, 'D'], [4, 'E'], [5, 'A'], [5, 'B'], [5, 'C'], [5, 'D'], [5, 'E']]\r\n>>>\r\n<\/code><\/pre>\n>>> iter_string = \"some text\"\r\n>>> comp_list = [x for x in iter_string if x !=\" \"]\r\n>>> print(comp_list)\r\n['s', 'o', 'm', 'e', 't', 'e', 'x', 't']\r\n<\/code><\/pre>\n>>> dict_comp = {x:chr(65+x) for x in range(1, 11)}\r\n>>> type(dict_comp)\r\n<class 'dict'>\r\n>>> print(dict_comp)\r\n{1: 'B', 2: 'C', 3: 'D', 4: 'E', 5: 'F', 6: 'G', 7: 'H', 8: 'I', 9: 'J', 10: 'K'}\r\n<\/code><\/pre>\n>>> set_comp = {x ** 3 for x in range(10) if x % 2 == 0}\r\n>>> type(set_comp)\r\n<class 'set'>\r\n>>> print(set_comp)\r\n{0, 8, 64, 512, 216}\r\n<\/code><\/pre>\nWhen to Use List Comprehensions in Python<\/h2>\n
>>> customers = [{\"is_kyc_passed\": False}, {\"is_kyc_passed\": True}]\r\n>>> kyc_results = []\r\n>>> for customer in customers:\r\n... kyc_results.append(customer[\"is_kyc_passed\"])\r\n...\r\n>>> all(kyc_results)\r\nFalse<\/code><\/pre>\n>>> customers = [{\"is_kyc_passed\": False}, {\"is_kyc_passed\": True}]\r\n>>> all(customer[\"is_kyc_passed\"] for customer in customers)\r\nFalse<\/code><\/pre>\nBenefits of Using List Comprehensions<\/h2>\n
\nFor a better understanding of what benefits list comprehensions brings to Python developers<\/a>, one can also pay attention to the following:<\/p>\n\n
Difference Between Iterable and Iterator<\/h2>\n
\nIterable is a “sequence” of data, you can iterate over using a loop. The easiest visible example of iterable can be a list of integers – [1, 2, 3, 4, 5, 6, 7]. However, it\u2019s possible to iterate over other types of data like strings, dicts, tuples, sets, etc.
\nBasically, any object that has iter()<\/b> method can be used as an iterable. You can check it using hasattr()<\/b> function in the interpreter.<\/p>\n>>> hasattr(str, '__iter__')\r\nTrue\r\n>>> hasattr(bool, '__iter__')\r\nFalse\r\n<\/code><\/pre>\n\n
>>> simple_list = [1, 2, 3]\r\n>>> my_iterator = iter(simple_list)\r\n>>> print(my_iterator)\r\n<list_iterator object at 0x7f66b6288630>\r\n>>> next(my_iterator)\r\n1\r\n>>> next(my_iterator)\r\n2\r\n>>> next(my_iterator)\r\n3\r\n>>> next(my_iterator)\r\nTraceback (most recent call last):\r\nFile \"<stdin>\", line 1, in <module>\r\nStopIteration\r\n<\/code><\/pre>\nGenerator Expressions<\/h2>\n
\nThe main feature of generator expression is evaluating the elements on demand. When you call a normal function with a return statement the function is terminated whenever it encounters a return statement. In a function with a yield<\/b> statement the state of the function is \u201csaved\u201d from the last call and can be picked up the next time you call a generator function.<\/p>\n>>> def my_gen():\r\n... for x in range(5):\r\n... yield x\r\n<\/code><\/pre>\n
\nGenerator expression in Python allows creating a generator on a fly without a yield<\/b> keyword. However, it doesn\u2019t share the whole power of generator created with a yield function. The syntax and concept is similar to list comprehensions:<\/p>\n>>> gen_exp = (x ** 2 for x in range(10) if x % 2 == 0)\r\n>>> for x in gen_exp:\r\n... print(x)\r\n4\r\n16\r\n36\r\n64\r\n<\/code><\/pre>\n>>> list_comp = [x ** 2 for x in range(10) if x % 2 == 0]\r\n>>> gen_exp = (x ** 2 for x in range(10) if x % 2 == 0)\r\n>>> print(list_comp)\r\n[0, 4, 16, 36, 64]\r\n>>> print(gen_exp)\r\n<generator object <genexpr> at 0x7f600131c410>\r\n<\/code><\/pre>\n
\nNote:<\/b> in Python 2 using range()<\/strong> function can’t actually reflect the advantage in term of size, as it still keeps the whole list of elements in memory. In Python 3, however, this example is viable as the range()<\/strong> returns a range object.<\/p>\n>>> from sys import getsizeof\r\n>>> my_comp = [x * 5 for x in range(1000)]\r\n>>> my_gen = (x * 5 for x in range(1000))\r\n>>> getsizeof(my_comp)\r\n9024\r\n>>> getsizeof(my_gen)\r\n88\r\n<\/code><\/pre>\n
\nOn the other hand, generator will be slower, as every time the element of sequence is calculated and yielded, function context\/state has to be saved to be picked up next time for generating next value. That “saving and loading function context\/state” takes time.
\nNote:<\/b> Of course, there are different ways to provide Python ‘generator to list’ conversion, besides initial using square brackets where Python generates lists via list comprehension. If it’s necessary to convert a generator to a list, Python developers can use, for example, the list()<\/b> function or the unpack operator *<\/b>.<\/p>\n