seminars.fb

Seminar (Fri Oct 1): “All the Syntax They Didn’t Teach You In School”

Title	All the Syntax They Didn’t Teach You In School
Topic	Overview of Contemporary and Upcoming Syntax in Python
Date	Fri Oct 1
Keywords	comprehensions, unpacking, floor division, true division, augmented assignment, walrus, pattern matching, string formatting, f-string, breakpoint, dataclasses, class decorators, metaclasses, __init_subclass__

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill-levels.

Our expected audience should comprise attendees with a…

• working familiarity with the built-in types in Python (dict, list, tuple, set)
• familiarity with core Python syntax
• (optional) familiarity with the differences between Python 2 and Python 3

During this session, we will endeavour to guide our audience to developing…

• familiarity with new syntax, approaches, and concepts introduced in Python 3.0, Python 3.6, and later
• familiarity with syntax coming in future version of Python
• understanding of the design of Python, what constitutes contemporary Python, and how these fit into a coherent vision for the language

Abstract

In previous seminars, we’ve seen code that makes use of syntax, functionality, and features introduced in Python 3.0 or Python 3.6 or later versions. These have been presented without drawing any special attention to them, but care has been taken to ensure that all code samples have been written in a modern style.

In this seminar, we’ll take a look at a handful of examples of new functionality or features that constitute modern Python style. But instead of parading these one-after-the-other, we’ll take a look at the underlying design considerations that tie these together, in an attempt to convey an over-arching, coherent understanding of what constitutes modern, fluent Python.

Sample Agenda:

• comprehension syntax
  • PEP-202: List Comprehensions
  • PEP-275; Dict Comprehensions
  • PEP-530: Asynchronous Comprehensions
• unpacking syntax
  • PEP-448: Additional Unpacking Generalizations
  • PEP-3132: Extended Iterable Unpacking
  • PEP-3113: Removal of Tuple Parameter Unpacking
  • PEP-584: Add Union Operators to dict
• division
  • PEP-238: Changing the Division Operator
• augmented assignment
  • PEP-572: Assignment Expressions
• matching
  • PEP-634: Structural Pattern Matching
• arguments
  • PEP-3102: Keyword-Only Arguments
  • PEP-457: Positional-Only Parameters
• formatting
  • PEP-3101: Advanced String Formatting
  • PEP-498: Literal String Interpolation
• debugging
  • PEP-553: built-in breakpoint()
• object orientation
  • PEP-557: Data Classes
  • PEP-3129: Class Decorators
  • PEP-3115 Metaclasses in Python 3000
  • PEP-487: Simpler Customisation of Class Creation

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you use Python more and more in your work?

In a future seminar, we can take a look at these areas in isolation and discuss the design problems these solve, as well as the overall thematic direction Python is taking as it evolves.

We can discuss:

• the focus on iteration and __iter__ as a core protocol in Python (and a partner to __call__); and the connection to asyncio and the new async/await syntax
• the focus on eliminating ambiguities; how fluent use of Python can lead to code that is more specific, less ambiguous, less error-prone, and more robust (in the absence of even over mechanisms like PEP-484 type hinting!)
• how approaches to object orientation differ in Python, and how contemporary Python has leaned into

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Notes

print("Let's get started!")

Modes of Division, Ambiguity, Local Reasoning, & PEP-238

(PEP-238: Changing the Division Operator)[https://www.python.org/dev/peps/pep-0238/] (Python 2.2)

print '5  / 2  =', 5  / 2  # int / int -> int
print '5. / 2  =', 5. / 2  # float / int -> float
print '5  / 2. =', 5  / 2. # int / float -> float
print '5. / 2. =', 5. / 2. # float / float -> float

def f(x, y):
    return x / y

# -------------- #

print 'f(5  / 2 ) =', f(5  / 2 )
print 'f(5. / 2 ) =', f(5. / 2 )
print 'f(5  / 2.) =', f(5  / 2.)
print 'f(5. / 2.) =', f(5. / 2.)

x, y = 5, 2
z = f(x, y)
...

# Modes of Division:
#    “true” — return actual (possibly fractional) result
#    “truncating” — return result with fractional part truncated
#    “floor” — return result rounded down (toward -∞)

def f(x, y):
    return x / y # explicit “true” division

def g(x, y):
    return x // y # explicit “floor” division

print(f'{f(5 , 2 ) = }   {g(5 , 2 ) = }')
print(f'{f(5., 2 ) = }   {g(5., 2 ) = }')
print(f'{f(5 , 2.) = }   {g(5 , 2.) = }')
print(f'{f(5., 2.) = }   {g(5., 2.) = }')

xs = [1, 2, 3, 4, 5, 6]
print(f'{xs[len(xs) // 2] = }')
print(f'{xs[len(xs) * 1.0 // 2] = }')

Joining Things & PEP-448

def f(x, y):
    return float(x) / y

def f(xs, ys):
    return list(xs) + list(ys)

print(f'{f([1, 2, 3], [4, 5, 6]) = }')
print(f'{f((1, 2, 3), (4, 5, 6)) = }')
#  print(f'{f({1, 2, 3}, {4, 5, 6}) = }')
#  print(f'{f((1, 2, 3), [4, 5, 6]) = }')
#  print(f'{f([1, 2, 3], (4, 5, 6)) = }')

PEP-448: Additional Unpacking Generalizations (Python 3.5)

xs, ys = [1, 2, 3], [4, 5, 6]

zs = [*xs, *ys]
print(f'{zs = }')

zs = {*xs, *ys}
print(f'{zs = }')

zs = *xs, *ys
print(f'{zs = }')

def f(xs, ys):
    return {*xs, *ys}    

PEP-584: Add Union Operators To dict (Python 3.9)

from collections import ChainMap
from itertools import chain

d1 = {'one': 'uno', 'two': 'dos'}
d2 = {              'two': 'deux', 'three': 'trois'}

d3 = d1.copy()
d3.update(d2)
print(f'{d3 = }')

d3 = dict(chain(d1.items(), d2.items()))
print(f'{d3 = }')

d3 = dict(ChainMap(d2, d1))
print(f'{d3 = }')

d3 = {**d1, **d2, 'four': 'vier'}
print(f'{d3 = }')

d3 = d1 | d2
print(f'{d3 = }')

Unpacking, Readability, & PEP-3132

PEP-3132: Extended Iterable Unpacking (Python 3.0)

t = 1, 2
a, b, c = t

t = 1, 2, 3, 4, 5, 6, 7, 8
a, b, *_ = t

def compute(*_, **__): pass

for _ in range(10):
    pass

for some_really_important_variable in range(10):
    ...
    ...
    ...
    ...
    ...
    ...
    ...
    ...
    ...

for _ in range(10):
    ...
    ...
    ...
    ...
    ...
    ...

from random import randint
data = [randint(0, 10) for _ in range(100)]

#  head, *_ = data # `data[0]`
#  *_, tail = data # `data[-1]`

head, *_, tail = data # NOTE: not same as `head, tail = data[0], data[-1]`

Design, APIs, & PEP-3102 & PEP-570

PEP 3102: Keyword-Only Arguments (Python 3.0)

def f(data, flag, mode):
    pass

def f(data, dry_run, use_prod):
    pass

f(..., True, False)
f(..., False, True)

f(..., dry_run=True, use_prod=False)

def f(data, *, dry_run, use_prod):
    pass

f(..., dry_run=True, use_prod=False)

def f(a, *, b, c):
    pass

def f(a, *, c, b):
    pass

f(10, b=200, c=3_000)

def f(data, mode):
    pass

def f(data, *, mode):
    pass

PEP 570: Notation For Positional-Only Parameters (Python 3.8)

def f(a, /, b, c):
    pass

f(10, 200, 3_000)
f(10, b=200, c=3_000)

help(len)
len([1, 2, 3])
len(obj=[1, 2, 3])

def f(some_stuff, /, b, c):
    pass

def f(raw_data, /, b, c):
    pass

f(10, 200, 3_000)

Multiplying & PEP-465

PEP-465: A dedicated infix operator for matrix multiplication (Python 3.5)

from numpy import eye, matrix
from numpy.random import default_rng
rng = default_rng(0)

xs = rng.integers(10, size=(size := 3, size))
ys = eye(size, dtype=int)

print(
    xs,
    ys,
    xs * ys,
    #  matrix(xs) * matrix(ys),
    xs @ ys,
    sep='\n',
)

Dividing, Path-Concatenation, & PEP-428

PEP 428: The pathlib module – object-oriented filesystem paths (Python 3.4)

input_filename = '/tmp/data-file.dat'
#  output_filename = input_filename + '.json'
#  output_filename = input_filename.replace('dat', 'json')
output_filename = input_filename.replace('.dat', '.json')

print(f'{input_filename  = }')
print(f'{output_filename = }')

from pathlib import Path
input_filename = Path('/tmp/data-file.dat')
output_filename = input_filename.with_suffix('.json')

print(f'{input_filename  = }')
print(f'{output_filename = }')

print(f'{output_filename.parent = }')
print(f'{output_filename.suffix = }')
print(f'{output_filename.name   = }')
print(f'{output_filename.stem   = }')

d = Path('/tmp')
print(f'{d / "abc" / "xyz" / output_filename.name = }')

Variadic Arguments & Argument Unpacking

print(1, 2, 3)
min(1, 2, 3)
zip('abc', range(3), [4, 5, 6])

def f(*args):
    print('f'.center(30, '='))
    print(f'{args = }')

f()
f(1, 2)
f(1, 2, 3, 4, 5, 6, 7)

def f(**kwargs):
    print('f'.center(30, '='))
    print(f'{kwargs = }')

f()
f(a=1, b=2)
f(a=1, b=2, c=3)

def f(a, b, c):
    pass

data = [1, 2, 3]
f(data[0], data[1], data[2])
f(*data)

def f(a, b, c):
    pass

data = {'a': 1, 'b': 2, 'c': 3}
f(a=data['a'], b=data['b'], c=data['c'])
f(**data)

print(f'{min( 1, 2, 3 ) = }')
print(f'{min([1, 2, 3]) = }')

def f(*data):
    print(f'{data = }')

f(*[1, 2, 3])

def f(data):
    print(f'{data = }')

f([1, 2, 3])

obj = 123

print(f'{isinstance(obj, int) = }')
print(f'{isinstance(obj, (int, float)) = }')

obj = 123

TYPES = {int, float}
#  print(f'{isinstance(obj, TYPES) = }')
print(f'{isinstance(obj, (*TYPES,)) = }')

def isinstance(obj, *types):
    for t in types:
        if isisintance(obj, t): return True

isinstance(obj, int)
isinstance(obj, int, float)

from random import choice as py_choice
from numpy.random import choice as np_choice
print(f"{py_choice('abcdef') = }")
print(f"{np_choice('abcdef') = }")

def core(a, b, c, d):
    pass

def convenience(a, b, d):
    return core(a, b, c=b, d=d)

# ----- #

convenience(..., ...) # -- user

def core(a, b, c, d=None):
    pass

def convenience(a, b, *args, **kwargs):
    return core(a, b, *args, c=True, **kwargs)

convenience(..., ...)
convenience(..., ..., d=...)

Readability, Comprehensions, & PEP-202 & PEP-274

PEP 202: List Comprehensions (Python 2.0)

PEP 274: Dict Comprehensions (Python 2.7, Python 3.0)

xs = [1, 2, 3, 4, 5]

ys = []
for x in xs:
    ys.append(x**2)

zs = [x**2 for x in xs]

print(f'{xs = }')
print(f'{ys = }')
print(f'{zs = }')

xs = [1, 2, 3, 4, 5]

ys = []
for x in xs:
    if x % 2 == 0:
        ys.append(x**2)

zs = [ x**2 for x in xs if x % 2 == 0 ]

print(f'{xs = }')
print(f'{ys = }')
print(f'{zs = }')

xs = [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5]

ys = [0]
for x in xs:
    if x % 2 == 0:
        ys.append(x**2 + ys[-1])

zs = [0] + [ x**2 + zs[-1] for x in xs if x % 2 == 0 ]
#                   ------

print(f'{xs = }')
print(f'{ys = }')
#  print(f'{zs = }')

xs = range(3)
...
...
...
result = [x**2 for x in xs if x % 2 == 0]
...
...
...

data = range(3)

for x in data:
    print(x)

#  [print(x) for x in data]

xs = [-3, -2, -1, 0, 1, 2, 3]

lc = [x**2 for x in xs]
sc = {x**2 for x in xs}
dc = {x: x**2 for x in xs}

print(f'{lc = }')
print(f'{sc = }')
print(f'{dc = }')

xs = [-3, -2, -1, 0, 1, 2, 3]

ge = (x**2 for x in xs)
tc = *(x**2 for x in xs),

print(f'{ge = }')
print(f'{tc = }')

from collections import defaultdict

hosts = [
    'abc.net',
    'def.net',
    'xyz.com',
]

by_tld = defaultdict(set)
for h in hosts:
    tld = h.rsplit('.', 1)[-1]
    by_tld[tld].add(h)

print(
    by_tld,
)

Text, Formatting, PEP-3101 & PEP-498

PEP-3101: Advanced String Formatting (Python 3.0)

PEP-498: Literal String Interpolation (Python 3.6)

x = 123.456

print('x = %s' % x)
print('x = %.1f' % x)
print('x = %d' % x)
print('x = %r' % x)

xs = [
        123.123,
    456_456.4,
        789.789,
]
for x in xs:
    print('%10,.1f' % x)

from datetime import datetime

now = datetime.now()

print('%s' % now)
print('%r' % now)

from datetime import datetime

TEMPLATE = '''
Report
======
Date: %(date)s
Data: %(data)s
'''.strip()

now = datetime.now()
data = [1, 2, 3]
print(
    #  TEMPLATE % {'date': now, 'data': data},
    TEMPLATE % {'date': now.strftime('%a %b %d, %Y'), 'data': data},
)

from datetime import datetime

TEMPLATE = '''
Report
======
Date: {date}
Data: {data}
'''.strip()

now = datetime.now()
data = [1, 2, 3]
print(
    TEMPLATE.format(date=now.strftime('%a %b %d, %Y'), data=data),
)

from datetime import datetime

TEMPLATE = '''
Report
======
Date: {date:%a %b %d, %Y}
Data: {data!r:>16}
'''.strip()

now = datetime.now()
data = [1, 2, 3]
print(
    TEMPLATE.format(date=now, data=data),
)

from datetime import datetime

now = datetime.now()
print(f'{now:%a %b %d, %Y}')

x, y = 10, 20
print(f'{x +  y = :>5}')
print(f'{x -  y = :>5}')
print(f'{x *  y = :>5}')
print(f'{x /  y = :>5}')
print(f'{x // y = :>5}')

Taking a Break & PEP-553

PEP-553: Built-in breakpoint() (Python 3.7)

def f(x, y):
    return g(x, y - 1)

def g(x, y):
    return h(x, y - 1)

def h(x, y):
    return x / (y - 1)

f(1, 3)

from pdb import post_mortem

def f(x, y):
    return g(x, y - 1)

def g(x, y):
    return h(x, y - 1)

def h(x, y):
    return x / (y - 1)

try:
    f(1, 3)
except Exception:
    post_mortem()

from pdb import set_trace

def f(x, y):
    return g(x, y - 1)

def g(x, y):
    return h(x, y - 1)

def h(x, y):
    set_trace()
    return x / (y - 1)

f(1, 3)

from pdb import set_trace
import sys; sys.breakpointhook = set_trace

def f(x, y):
    return g(x, y - 1)

def g(x, y):
    return h(x, y - 1)

def h(x, y):
    breakpoint()
    return x / (y - 1)

f(1, 3)

from inspect import currentframe, getouterframes
def breakpointhook():
    for fr in getouterframes(currentframe())[1:-1]:
        print(f'line {fr.lineno} <{fr.function}>: {fr.frame.f_locals}')

import sys; sys.breakpointhook = breakpointhook

def f(x, y):
    return g(x, y - 1)

def g(x, y):
    return h(x, y - 1)

def h(x, y):
    breakpoint()
    return x / (y - 1)

f(1, 3)

Assignment & PEP-572

PEP 572: Assignment Expressions (Python 3.8.)

x = 123

if x == 123:
    pass

from re import compile as re_compile

abcd_pattern = re_compile('ab(.)d')
wxyz_pattern = re_compile('wx(.)z')

data = '''
wxyz
abcd
ab¢d
'''.strip().splitlines()

for line in data:
    mo = abcd_pattern.fullmatch(line)
    if mo:
        print(mo.group(1))
    mo = wxyz_pattern.fullmatch(line)
    if mo:
        print(mo.group(1))

from re import compile as re_compile

patt0 = re_compile('ab(.)d wx(.)z')
patt1 = re_compile('a(..)d')
patt2 = re_compile('(.).+')

data = '''
abcd wxyz
abcd
ab¢d
'''.strip().splitlines()

for line in data:
    mo = patt0.fullmatch(line)
    if mo:
        print(mo.groups())
    mo = patt1.fullmatch(line)
    if mo:
        print(mo.groups())
    mo = patt2.fullmatch(line)
    if mo:
        print(mo.groups())

from re import compile as re_compile

patt0 = re_compile('ab(.)d wx(.)z')
patt1 = re_compile('a(..)d')
patt2 = re_compile('(.).+')

data = '''
abcd wxyz
abcd
ab¢d
'''.strip().splitlines()

for line in data:
    mo = patt0.fullmatch(line)
    if mo:
        print(mo.groups())
    else:
        mo = patt1.fullmatch(line)
        if mo:
            print(mo.groups())
        else:
            mo = patt2.fullmatch(line)
            if mo:
                print(mo.groups())

from re import compile as re_compile

patt0 = re_compile('ab(.)d wx(.)z')
patt1 = re_compile('a(..)d')
patt2 = re_compile('(.).+')

data = '''
abcd wxyz
abcd
ab¢d
'''.strip().splitlines()

for line in data:
    if mo := patt0.fullmatch(line):
        print(mo.groups())
    elif mo := patt1.fullmatch(line):
        print(mo.groups())
    elif mo := patt2.fullmatch(line):
        print(mo.groups())

from time import sleep

def slow(x):
    sleep(.1)
    return x ** 2

xs = [1, 2, 3, 4, 5]
ys = [slow(x) for x in xs if slow(x) % 2 == 0]

print(f'{xs = }')
print(f'{ys = }')

from time import sleep

def slow(x):
    sleep(.1)
    return x ** 2

xs = [1, 2, 3, 4, 5]
ys = [res for x in xs if (res := slow(x)) % 2 == 0]

print(f'{xs = }')
print(f'{ys = }')

Q & A

class context:
    def __enter__(self):
        pass
    def __exit__(self, *_):
        pass


def build_class(*_, **__):
    pass