seminars.fb

Python @ Facebook (seminars.fb.notes.dutc.io)

Hardware Engineering Python Learning Series

Newsletter

Why should you sign up for our newsletter?

1. Get Python tips & tricks, brainteasers, code reviews, and other exclusive, original technical content from Don’t Use This Code!
2. Stay up-to-date with new developments in the projects you love!
3. Become aware of upcoming events in the global open source community!
4. Receive advance notice and special deals on new offerings from Don’t Use This Code!

Do you want to be a Python expert? Sign up today @ bit.ly/expert-python!

Sign up!

today (Fri Oct 28): “Data Analysis with Python”
tmate link (https://tmate.io/t/dutc/seminars-fb)

About

Don’t Use This Code; Training & Consulting

Don’t Use This Code is a professional training, coaching, and consulting company. We are deeply invested in the open source scientific computing community, and are dedicated to bringing better processes, better tools, and better understanding to the world.

Don’t Use This Code is growing! We are currently seeking new partners, new clients, and new engagements for our expert consulting and training services.

Teams looking to better employ these tools would benefit from our wide range of training courses on offer, ranging from an intensive introduction to Python fundamentals to advanced applications of Python for building large-scale, production systems. Working with your team, we can craft targeted curricula to meet your training goals. We are also available for consulting services such as building scientific computing and numerical analysis systems using technologies like Python and React.

We pride ourselves on delivering top-notch training. We are committed to providing quality training, and we do so by investing in three key areas: our content, our processes, and our contributors.

James Powell; Consultant, Instructor, & Presenter

James Powell is a professional Python programmer and enthusiast. He got his start with the language by building reporting and analysis systems for proprietary trading offices; now, he uses his experience as a consultant for those building data engineering and scientific computing platforms for a wide range of clients using cutting-edge open source tools like Python and React.

He also currently serves as a Board Director, Chair, and Vice President at NumFOCUS, the 501(c)3 non-profit that supports all the major tools in the Python data analysis ecosystem (i.e., pandas, numpy, jupyter, matplotlib). At NumFOCUS, he helps build global open source communities for data scientists, data engineers, and business analysts. He helps NumFOCUS run the PyData conference series and has sat on speaker selection and organizing committees for 18 conferences. James is also a prolific speaker: since 2013, he has given over seventy (70) conference talks at over fifty (50) Python events worldwide.

Table of Contents

date	session
Fri Oct 28, 2022	workshop	notes & materials
Fri Oct 14, 2022	seminar	notes & materials
Fri Sep 16, 2022	seminar	notes & materials
Fri May 6, 2022	seminar	notes & materials
Fri Feb 11, 2022	seminar	notes & materials
Fri Dec 17, 2021	seminar	notes & materials
Fri Dec 3, 2021	seminar	notes & materials
Fri Nov 5, 2021	seminar	notes & materials
Fri Oct 15, 2021	seminar	notes & materials
Fri Oct 1, 2021	seminar	notes & materials
Fri May 21, 2021	workshop	notes & materials
Fri May 7, 2021	seminar	notes & materials
Fri Mar 12, 2021	seminar	notes & materials
Fri Dec 18, 2020	seminar	notes & materials
Fri Dec 18, 2020	discussion	notes & materials
Fri Dec 11, 2020	discussion	notes & materials
Fri Dec 4, 2020	seminar	notes & materials
Wed Nov 18, 2020	seminar	notes & materials
Wed Nov 11, 2020	discussion	notes & materials
Wed Nov 11, 2020	seminar	notes & materials
Fri Nov 6, 2020	seminar	notes & materials
Mon Oct 30, 2020	discussion	notes & materials
Fri Oct 23, 2020	seminar	notes & materials
Mon Oct 12, 2020	discussion	notes & materials
Mon Oct 12, 2020	seminar	notes & materials
Fri Oct 9, 2020	seminar	notes & materials

Notes & Materials

Workshop (Fri Oct 28): “Data Analysis”

Title	“Data Analysis”
Topic	hard problems in data analysis
Date	Fri Oct 28
Time	1pm~3pm PST
Keywords	pandas

Materials

• Notes

In this advanced workshop, we will share with you some of the most valuable and most complex problems problems we have had to solve in pandas as part of our work with the Hardware Insights team.

In this workshop, we’ll work through these problems together, to see how we can make very sophisticated analyses straightforward and obvious.

Specifically, we will look to answering the following questions:

• Given a pandas.Series containing snapshots of some evolving process, how do we determine the state at any given point in time (such that we can index align this result against other structures)? How do we derive state at any given point in time given a pandas.Series containing deltas?
• Given multiple pandas.Series and pandas.DataFrame with MultiIndexes with overlapping levels, how do we perform index aligned operations to solve complex nested problems without explicit Python-level loops?
• Given a pandas.Series containing some measurement, how do we determine where the measurement has changed such that we can .groupby consecutive identical measurements?
• Given a pandas.Series or pandas.DataFrame with a rolling window operation, what do we do if the window sizes are so small that our “user defined functions” (“UDF”s) are effectively operating at Python speed? How do we drop down into raw numpy to restore lost performance when the operation can be pushed into the “restricted computation domain” (and what limits pandas from being amenable to these optimisations?)
• Given a pandas.Series or pandas.DataFrame with an expanding window operation with a “user defined function” (“UDF”) where the UDF will necessarily be called once per row, what alternate approaches can allow us to perform these operations efficiently?
• How do we decompose pandas.DataFrame.groupby operations that require operating on multiple columns without using pandas.DataFrame.groupby.apply? (e.g., how do we compute weighted averages with pandas.DataFrame.groupby.apply?)
• How do we jam everything into one line in pandas using .loc[lambda x: x], .pipe, .assign, and PEP-572 assignment expressions (:=)? Why might we actually choose to do this?
• How do we use numpy.meshgrid and numpy.mgrid together with pandas to construct structures containing the Cartesian product of all entities compared against all other entities?
• Given a pandas.Series containing a signal, how do we determine the maximum number of times the signal consecutively appears (by number of measurements or distance from minimum to maximum index value)?
• How do we perform operations in pandas on intervals of arbitrary convention (i.e., open/open, closed/closed, or open/closed) using pandas.cut?
• How do we select cross sections of pandas.Series or pandas.DataFrame with MultiIndex using pandas.IndexSlice? How do we perform exact entity selections with .loc and constructed MultiIndexes?

Seminar (Fri Oct 14): “The Standard Library”

Title	“The Standard Library”
Topic	a quick tour of the Python standard library
Date	Fri Oct 14
Time	10am~11am PST
Keywords	Python, the standard library

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill-levels.

Our expected audience should comprise attendees with a…

• basic familiarity with Python for basic scripting and analysis tasks.
• basic familiarity with pandas for simple data analysis

During this session, we will endeavor to guide our audience to developing…

• strong proficiency with use of pandas for time series analysis.

Abstract

In previous seminars, we’ve made thorough use of the Python standard library, and we’ve talked about what it takes to develop greater fluency and greater capabilities with Python.

In this seminar, we’ll take a closer look at the Python standard library. We’ll present a motivation for coming to terms with the standard library’s scope and size. We’ll present a perspective on how to understand where and when to use the standard library, rather than writing your own approach or introducing a third-party dependency. We’ll also tour the most interesting and useful parts of the standard library—parts which you could be using right now to improve your work!

What’s Next

Did you enjoy this seminar? Did you learn something new that will help you as you as you write your own pandas analyses?

In a future seminar, we can do a deeper dive into data structures and algorithms that can be applied to solving problems in your work. We can also tour major third party libraries that can bring immediate benefit to your code, and highlight tools that you might not be aware of.

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Sep 16): “Time Series Analysis in Python (with pandas!)”

Title	“Time Series Analysis with Python”
Topic	analysing common time series data using Python
Date	Fri Sep 16
Time	10am~11am PST
Keywords	Python, pandas, time series analysis

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill-levels.

Our expected audience should comprise attendees with a…

• basic familiarity with Python for basic scripting and analysis tasks.
• basic familiarity with pandas for simple data analysis

During this session, we will endeavor to guide our audience to developing…

• strong proficiency with use of pandas for time series analysis.

Abstract

In previous seminars, we have seen the use of pandas for analysis of columnar data, and the importance of understanding core NumPy and pandas concepts to make these analyses flexible, efficient, and fluent.

In this seminar, we will take a closer look at analyses involving time data. We’ll construct a sample dataset very similar to actual data that you might work with from Netpipeline, SeRF, ODS, or similar. We’ll investigate a number of specific, actionable analyses (i.e., analyses which tie directly to a business outcome or business decision,) and see how to construct them, step-by-step, in pandas

What’s Next

Did you enjoy this seminar? Did you learn something new that will help you as you as you write your own pandas analyses?

In a future semnar, we can do deeper into advanced pandas features and mechanics, such as the use of MultiIndex; transformation operations like .stack, .unstack, .melt, .pivot, and .pivot_table; grouping and windowing analyses using .groupby, .rolling, and .expanding; efficiency with large data sets (and the occassional need to drop down to the NumPy layer); or other topics that would help you make better use of this powerful analytical tool!

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri May 6): “Mastering the Basics of Python” (developing mastery of basic Python syntax and functionality)

Title	“Mastering the Basics of Python”
Topic	developing mastery of basic Python syntax and functionality
Date	Fri May 6
Time	10am~11am PST
Keywords	Python, the built-in data types, the built-in functions, the standard library, advanced syntax

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill-levels.

Our expected audience should comprise attendees with a…

• basic familiarity with Python for basic scripting and analysis tasks.

During this session, we will endeavor to guide our audience to developing…

• strong proficiency with the core syntax of Python.
• strong proficiency with the core functionality of Python, including the use of the standard library and the builtins (types and functions.)

Abstract

In previous seminars, we have used all manner of Python syntax and core functionality to demonstrate points about data analysis and software development. We have not called attention to some of the precise choices made in our code samples, preferring instead to discuss the use-case or theoretical topic at hand.

In this seminar, we will dive into some of the exacting, precise choices that we regularly make when writing even very simple pieces of Python code. While many of the topics we will discuss could be classified as “introductory” Python, we will approach them from the perspective of someone who has already written a good deal of code in Python, someone who is looking to revisit and solidify decisions they may subconsciously make every day in their code.

Sample Agenda:

• the builtin types:
  • what is the difference between a list and a tuple (and why is mutability/immutability the least interesting aspect)
  • what is a dict, what is a collections.defaultdict, and what is a collections.Counter, and what is a pandas.Series; how are they similar, how do they differ, and how do they solve subtly different problems?
  • what is a numpy.ndarray and how does it conceptually differ from a list?
  • what are the set and frozenset types?
  • what does it mean for something to be hashable? (why does this not require immutability but is commonly associated with it?)
• the builtin functions:
  • what is the key= argument, how do we use it, why is it preferrably to the Decorate-Sort-Undecorate/“Schwarzian Transform” formulation? (and what is the model for doing sorts, mins, and maxes in pandas/NumPy with a custom predicate?)
  • what is the id function, and how can we use it to better understand whether we are working with a view or a copy? (what is the difference between early/late-binding? what is the difference between live/snapshot views? how do we understand these questions in the context of numpy or pandas?)
  • what is the iter and next function? what is the difference between an iterator and an iterable? why does this matter, and how can we use this knowledge effectively?
  • what are the map and filter functions? what is the itertools module? when might we use these instead of comprehension syntax?
• syntax:
  • what is lambda syntax, why is it useful, and what does it tell people who are reading our code?
  • what are keyword-only and positional-only arguments, where are they useful, and how to we use them to improve long-term maintainability of our code?
  • what is the difference between from module import f, import module, import module as mod, and from module import f as func?

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you as you write larger Python scripts and analyses and write libraries to empower your colleagues’ work.

In a future seminar, we can go deeper into new syntax added to Python ≥3.6, and new approaches to writing Python that have evolved in the past five years.

We can discuss…

• key recent additions to the Python standard library that reveal a new, contemporary way of structuring Python programmes (e.g., eschewing “value intermediation” for an “objects-first”/“GC-first” approach.)
• recent additions to the Python core language and standary library enabling static verification via type hints (PEP=484) and external tooling such as MyPy.
• the long history of functional programming approaches in Python, and how they fit into the contemporary structure of the language, its tools, and recent guidance from language core developers.

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Feb 11): “Give me your data analysis, stat!” (using pandas and statsmodels for statistical analysis of large data sets)

Title	“Give me your data analysis, stat!”
Topic	using scipy, numpy, and statsmodels for statistical analysis of large data sets
Date	Fri, Feb 11 2022
Time	10am~11am PST
Keywords	descriptive stats, inferential statistics, distributions, probability, modelling, statsmodels, scipy, numpy, pandas

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill-levels.

Our expected audience should comprise attendees with a…

• familiarity with basic descriptive statistics (e.g., means, medians, standard deviations)
• basic familiarity with numeric computing using Python (e.g., basic numpy, pandas, scipy usage)

During this session, we will endeavor to guide our audience to developing…

• proficiency in describing one or more data sets statistically and visually
• familiarity with the wide breadth of questions statistics enables one to ask a data set
• familiarity with exploring existing relationships among data sets and using those relationships to make predictions about unseen data

Abstract

In previous seminars, we have transformed, visualised, and analysed large data sets using pandas. These analyses have been presented with a focus on the syntax and structure of the computational tools, rather than on the underlying analytical tasks.

In this seminar, we will focus on the analysis itself, and show how to do basic descriptive and inferential statistics tasks in Python using pandas, numpy, scipy, and statsmodels. We will discuss the motivation for this use of statistics, how to formulate and answer hypotheses, and how to do basic correleation and prediction.

Sample Agenda:

• descriptive statistics
  • mean, median, modes: how to compute them and their intuitive meaning
  • “mvsk” (mean, variance/standard deviation, skew, kurtosis): how to compute them and their intuiitive meaning
  • PDFs, CDFs, and PMFs
  • the normal distribution
  • other common distributions: Binomial, Poisson, &c.
• hypotheses and testing
  • formulating hypothesis (from a frequentist perspective): H₀, H₁, α, p-values
  • common statistical tests: z-test, one- vs two-tailed tests, t-test, paired t-test, “ANOVA”
  • basic Bayesian statistics
  • laws of probability
• correlation and prediction
  • uncover relationships across multiple data sets
    • correlation (pearson, spearman)
  • predict expectations of new data from existing data
    • regression (linear, logistic)

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you analyse larger and larger data sets?

In a future seminar, we can go into greater depth on statistical modeling, inference, and prediction. We can also discuss how to approach statistics from the Bayesian approach, or tie our statistical & probability knowledge to topics in information theory.

We can discuss…

• the Bayesian Approach, how it meaningfully differs from the frequentist approach, and how to formulate common problems within this framework.
• common statistical distributions (Weibull, Exponential, F-, T-), their formulation, their associated tests, and our intuition around where and why these may appear in our datasets.
• advanced statistical modeling techniques to assess and improve prediction accuracy.

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Dec 17): “Don’t Forget About the .index!”

Title	Don’t Forget About the .index!
Topic	Fluent Use of pandas & core concepts
Date	Fri Dec 17
Keywords	pandas.Index, MultiIndex

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill-levels.

Our expected audience should comprise attendees with a…

• familiarity with analysis of columnar/tabular or time-series data (e.g., CSV data, Excel data.)
• basic familiarity with the use of pandas for columnar data analysis.

During this session, we will endeavour to guide our audience to developing…

• proficiency with index-aligned operations on pandas.Series and pandas.DataFrame objects.
• familiarity with the mechanics of the pandas.Index.

Abstract

In a previous seminar, we discussed data analysis of tabular/columnar data sets using pandas.

In this seminar, we will take a close look at what makes pandas special: the index! We’ll look at how a pandas.Series can be conceptualised as a mathematical “restricted computation domain” similar to the numpy.ndarray with some business information attached—a coördinate system or indexing system. We’ll talk about how a pandas.DataFrame is not a two-dimensional data structure but a doubly-indexed collection of index-aligned one-dimesional data. We’ll discuss the mechanisms employed by the Index as part of .loc operations, and core concepts of implicit/explicit hierarchy and monotonicity.

Agenda:

• index alignment
• the pandas.Index and .get_loc, indexing, monotonicity
• index types (incl. int64 vs Int64)
• .index vs .columns
• .T, .reindex, .set_index, .reset_index, .rename, .swapaxes, .reorder_levels, .swaplevel, .droplevel
• MultiIndex, explicit vs implicit heirarchy
• .melt, .pivot
• custom indices

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you use pandas more effectively, as you do more analysis in your work?

In a future seminar, we may dive deeper into the mechanics of pandas. We can:

• discuss the creation of custom indices (and how this ties into building libraries and frameworks that consume and return pandas.DataFrames)
• discuss extensions to pandas, such as subclassing pandas.Series and pandas.DataFrame and defining registered accessors
• discuss the “structural geometry” of a pandas.DataFrame and see how all operations can be described in-terms of manipulations of the .index, the .columns, and the ._data

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Dec 3): “Doing More with Asynchronous Python”

Title	Doing More with Asynchronous Python
Topic	Asynchronous Python with async/await
Date	Fri Dec 3
Keywords	generators, coroutines, await, async def, async for, async with, PEP-492

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill levels.

Our expected audience should comprise attendees with a…

• basic proficiency with the use of Python for synchronous scripting
• basic familiarity with concurrency mechanisms in general.

During this session, we will endeavour to guide our audience to developing…

• familiarity with how asyncio fits into the choices for concurrency mechanisms, including the advantages and limitations of each approach.
• proficiency with writing asynchronous code in Python, including the use of the asyncio library and new syntax added to Python 3.6.

Abstract

In a previous seminar, we gave a broad overview of concurrency approaches in Python, comparing them and contrasting them, touching briefly upon asyncio.

In this seminar, we will take a closer look at how asyncio approaches are structured. We’ll review the basics of generators and coroutines, build from there to asynchronous coroutines and event loops, discuss new syntax added to support asynchronous programming (e.g., async def/await.) We’ll also take a look at how this syntax fits into mechanisms added to the Python object model, and wrap up our discussion with a neat demo of a simple asynchronous system.

Agenda:

• Review: Generators & Coroutines
• (Review: Concurrency Mechanisms)
• Asyncio Mechanisms
• Asyncio Demo

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you write more complex Python systems that require concurrent approaches?

In a future seminar, we may dive deeper into asynchronous and concurrent design.

We can:

• visit the demo from this seminar and build it out more fully.
• consider how asynchronous approaches with asyncio combine with traditional concurrent approaches like threading and multiprocessing.
• do a deep dive into code using multiprocessing, including the use of multiprocessing.shared_memory, limitations of multiprocessing, and sophisticated use of entities such as subprocess.Popen.

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Nov 5): “Grouping, Rolling, and Expanding in pandas”

Title	Grouping, Rolling, and Expanding in pandas
Topic	Grouping (.groupby) and Window (.rolling, .expanding) Functions in pandas
Date	Fri Nov 5
Keywords	{DataFrame,Series}.groupby, .groupby.agg, .groupby.apply, .groupby.transform, .rolling, .expanding

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill levels.

Our expected audience should comprise attendees with a…

• basic proficiency with pandas
• basic familiarity with pandas.{Series,DataFrame}.groupby and grouping operations

During this session, we will endeavour to guide our audience to developing…

• (complete) proficiency with the use of windowing operations such as .groupby, .expanding, .rolling, and .ewm, including use with time series data and use of the corresponding Rolling, Window, Expanding, and ExponentialMovingWindow objects
• (basic) proficiency with fluent expression of complex combinations of windowing operations (e.g., .groupby(…).transform(lambda df: df.rolling(…))
• familiarity with performance considerations related to the use of common window functions

Abstract

In a previous seminar, we looked at the pandas.DataFrame and how it provides a data type for manipulating liked-indexed columnar datasets.

In this seminar, we will look at .groupby in-depth, as well as the various reduction operations it supports (.apply, .transform, .aggregate.) We will explore .groupby in the context of other window functions in pandas—functions which operate on “windows” of multiple rows or multiple columns to perform aggregations or other transformations.

We’ll also look at .rolling, .expanding, and .ewm, their various options and modalities, as well as the operations available on the Window, Rolling, Expanding, and ExponentialMovingWindow objects they return. We’ll discuss these operations in the context of time series analysis and discuss performance considerations related to the use of each.

Agenda:

• .groupby
  • .groupby.agg
  • .groupby.apply
  • .groupby.transform
• .rolling
• .expanding
• .ewm

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you use pandas and window functions more and more in your work?

In a future seminar, we may dive deeper into the use of pandas.

We can:

• cover .rolling with .tseries.offsets and other time series considerations in greater depth.
• discuss in-depth performance considerations, and the use of numba, cython or other tools to address performance issues with non-native window operations.
• discuss window indexers for defining custom window boundaries.

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Oct 15): “Seeing Things in Context with Context Managers”

Title	Seeing Things in Context with Context Managers
Topic	Resource Management & Context Managers
Date	Fri Oct 15
Keywords	context managers, with-statement, asynchronous context managers, contextlib, __del__, __weakref__, PEP-343, PEP-567, PEP-492

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill levels.

Our expected audience should comprise attendees with a…

• working familiarity with basic Python syntax.
• a desire to use more advanced features in Python like PEP-343 Context Managers.

During this session, we will endeavour to guide our audience to developing…

• complete familiarity with the metaphors and motivation behind the use of PEP-343 Context Managers
• complete familiarity with common use-cases and common built-in or standard library Context Managers
• proficiency in writing PEP-343 Context Managers (using contextlib.contextmanager)
• familiarity with topics related to Context Managers, including context-local state (PEP-567 Context Variables) and interaction with asyncio code (PEP-492 Asynchronous Context Managers)

Abstract

In previous seminars, we have discussed the motivation, mechanisms, and metaphors provided by advanced Python features such as generators, coroutines, and aspects of the OO model.

In this seminar, we will tackle PEP-343 Context Managers. We’ll discuss the motivation of deterministic management of resources, prior approaches and potential missteps and misapprehensions when coming from other programming languages. We’ll discuss the “metaphor” that context managers provide, and how this sequencing metaphor leads to a direct relationship between context managers and generators (typically via contextlib.contextmanager.) We’ll discuss the mechanism behind context managers, including the OO model API, as well as details related to context manager in asynchronous code (and the motivation and appropriate use of async with syntax.) Finally, we’ll discuss common problems related to composition of context managers, as well as the need for context-local state (and the subsequent development of PEP-567 context variables.)

Agenda:

• PEP-343: the with statement
• contextlib
• PEP-492: Coroutines with async and await syntax & asynchronous context managers
• PEP-567: context variables
• __del__ and weakref

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you use advanced features in Python more and more in your work?

In a future seminar, we can discuss other advanced Python syntax. We can discuss:

• the metaphors, mechanisms, and motivations behind decorators, including class decorators and higher-order decorators
• how to use context managers to design elegant, extensible APIs, as well as common compositional problems that users run into when designing around context managers.
• how the various asyncio-syntaxes interact (i.e., async def, async for, async with) including why there is no async while and why it is conceptually reasonable by mechanically impossible to yield from an asynchronous context manager.

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Seminar (Fri Oct 1): “All the Syntax They Didn’t Teach You In School”

Title	All the Syntax They Didn’t Teach You In School
Topic	Overview of Contemporary and Upcoming Syntax in Python
Date	Fri Oct 1
Keywords	comprehensions, unpacking, floor division, true division, augmented assignment, walrus, pattern matching, string formatting, f-string, breakpoint, dataclasses, class decorators, metaclasses, __init_subclass__

Materials

• Notes

Audience

These sessions are designed for a broad audience of non-software engineers and software programmers of all backgrounds and skill levels.

Our expected audience should comprise attendees with a…

• working familiarity with the built-in types in Python (dict, list, tuple, set)
• familiarity with core Python syntax
• (optional) familiarity with the differences between Python 2 and Python 3

During this session, we will endeavour to guide our audience to developing…

• familiarity with new syntax, approaches, and concepts introduced in Python 3.0, Python 3.6, and later
• familiarity with syntax coming in future versions of Python
• understanding of the design of Python, what constitutes contemporary Python, and how these fit into a coherent vision for the language

Abstract

In previous seminars, we’ve seen code that makes use of syntax, functionality, and features introduced in Python 3.0 or Python 3.6 or later versions. These have been presented without drawing any special attention to them, but care has been taken to ensure that all code samples have been written in a modern style.

In this seminar, we’ll take a look at a handful of examples of new functionality or features that constitute modern Python style. But instead of parading these one-after-the-other, we’ll take a look at the underlying design considerations that tie these together, in an attempt to convey an overarching, coherent understanding of what constitutes modern, fluent Python.

Sample Agenda:

• comprehension syntax
  • PEP-202: List Comprehensions
  • PEP-275; Dict Comprehensions
  • PEP-530: Asynchronous Comprehensions
• unpacking syntax
  • PEP-448: Additional Unpacking Generalizations
  • PEP-3132: Extended Iterable Unpacking
  • PEP-3113: Removal of Tuple Parameter Unpacking
  • PEP-584: Add Union Operators to dict
• division
  • PEP-238: Changing the Division Operator
• augmented assignment
  • PEP-572: Assignment Expressions
• matching
  • PEP-634: Structural Pattern Matching
• arguments
  • PEP-3102: Keyword-Only Arguments
  • PEP-457: Positional-Only Parameters
• formatting
  • PEP-3101: Advanced String Formatting
  • PEP-498: Literal String Interpolation
• debugging
  • PEP-553: built-in breakpoint()
• object orientation
  • PEP-557: Data Classes
  • PEP-3129: Class Decorators
  • PEP-3115 Metaclasses in Python 3000
  • PEP-487: Simpler Customisation of Class Creation

What’s Next?

Did you enjoy this seminar? Did you learn something new that will help you as you use Python more and more in your work?

In a future seminar, we can take a look at these areas in isolation and discuss the design problems these solve, as well as the overall thematic direction Python is taking as it evolves.

We can discuss:

• the focus on iteration and __iter__ as a core protocol in Python (and a partner to __call__); and the connection to asyncio and the new async/await syntax
• the focus on eliminating ambiguities; how fluent use of Python can lead to code that is more specific, less ambiguous, less error-prone, and more robust (in the absence of even over mechanisms like PEP-484 type hinting!)
• how approaches to object orientation differ in Python, and how contemporary Python has leaned into

If you’re interested in any of these topics, please let us know! Send us an e-mail at learning@dutc.io or contact us over Workplace with your feedback!

Advanced Use-Cases → “Graphs and Networks”

Workshop (Fri, May 21, 2021; 1:30 PM PST)

Theme: Graphs and networks

Topic: modeling graphs, graph problems, working with graphs in python

Keywords: nodes, edges, paths, graphs, networks, networkx

Presenter	James Powell james@dutc.io
Date	Friday, May 21, 2021
Time	1:30 PM PST

“All around me I see nodes and edges! Graphs are everywhere—in all sorts of problems we need to solve, from modeling connectivity in a data center to modeling workflows and business processes! In this session, we’ll brush up on some fundamental graph theory and discuss how to work with and model graphs in Python!”

Hands-on with Graph Analysis in Python.

Join Python expert-in-residence James Powell for a hands-on, small group workshop on Working with Graphs in Python! Using a case study as a foundation, we will cover topics based on attendees interests, such as:

• Using the Python library networkx
• Graph theory and algorithms: BFS, DFS, topological sorting
• Graph representations in Python
• Using Python generators
• Building libraries around graphs
• Integrating with numeric computing tooling, like numpy and pandas

There are no formal prerequisites, however, experience with Python is a must! There will be plenty of time during the session for questions and discussion.

Materials:

• Notes

---

Programming Fundamentals → “Metaprogramming in Python: Writing code that writes code”

Seminar (Fri, May 7, 2021; 12:30 PM PST)

Theme: Programming Fundamentals

Topic: Metaprogramming

Presenter	James Powell james@dutc.io
Date	Friday, May 7, 2021
Time	12:30 PM PST

Computers make our lives easier by allowing us to automate tasks we would otherwise do by hand. But the practice of writing code itself can be tedious. So why can’t I write a computer programme that automates my task of writing computer programmes that automates my task of writing computer programmes that automates my task of writing computer programmes…?

This seminar will present a view of metaprogramming approaches in Python, focusing on questions like:

• what is metaprogramming, and when can it be a productive approach for writing more robust code (and automating the process of writing boilerplate)?
• what does metaprogramming mean in Python, and why does it appear to be so closely tied with the object construction protocols?
• what metaprogramming techniques are you already using without even knowing?
• what is collections.namedtuple, dataclasses.dataclass, and enum.Enum and how do they represent three core metaprogramming approaches: eval/exec, def- and class-decorators, and metaclasses/init_subclass/build_class?
• what are update anomalies, and how can metaprogramming approaches help us to add programmatic structure (to our code!) to help avoid them?

Materials:

• Notes

---

Programming Fundamentals → “Complexity Analysis, Big-O, Data Structures, and Algorithms”

Seminar (Fri, Mar 12, 2021; 1 PM PST)

Theme: Programming Fundamentals

Topic: Complexity Analysis, Big-O, Data Structures, and Algorithms

Keywords: complexity analysis, big-o, data structures, and algorithms

Presenter	James Powell james@dutc.io
Date	Friday, March 12, 2021
Time	1:00 PM PST

Okay, so your code is slow. That’s no good! Nobody likes slow code. But what does it really matter? And when does it really matter? And what does it all really mean?

This seminar will present an introductory view of a critical topic in software development: complexity analysis and Big-O notation as it applies to our choices of algorithms and data structures in Python programmes. We will be covering important questions like:

• what it means for some code to be “slow”: is this a fundamental property of the problem itself—after all, some problems are genuinely hard—or is this some deficiency in our approach?
• how should we think about optimising our code? when should we micro-optimise and when should we think structurally?
• how can Big-O notation give us a way to discuss the “asymptotic” behaviour of our code; why is it important to talk about how the performance of code “scales” with problem size?
• how does our choice of built-in or user-defined data structures in Python have a direct impact on the performance of our code, and, consequentially, the underlying tractability of our problem-solving approach?
• how does our choice of algorithms tie directly into our choice of data structures and have direct impact on the performance of our work as it scales?

Materials:

• Notes

---

Programming Fundamentals → “Generators”

Seminar (Fri, Dec 18, 2020; 9 AM PST)

Theme: Programming Fundamentals

Topic: Generators

Keywords: generators, laziness, coroutines, comprehension syntax

Presenter	James Powell james@dutc.io
Date	Friday, December 18, 2020
Time	9:00 AM PST

Materials:

• Slides
• Notes

---

Programming Fundamentals → “OO Design”

Seminar (Fri, Dec 18, 2020; 11 PM PST)

Theme: Programming Fundamentals

Topic: OO Design

Keywords: objects, classes, design

Presenter	James Powell james@dutc.io
Date	Friday, December 18, 2020
Time	11:00 PM PST

Materials:

• Questions
• Game

---

Grab Bag → “Stocking Stuffers”

Seminar (Fri, Dec 11, 2020; 12 PM PST)

Theme: Grab Bag

Topic: Stocking Stuffers

Keywords: simulation design

Presenter	James Powell james@dutc.io
Date	Friday, December 11, 2020
Time	12:00 PM PST

Materials:

• Game
• Questions

---

Tools & Approaches → “designing functions”

Seminar (Fri, Dec 4, 2020; 9 PM PST)

Theme: Tools and Approaches

Topic: Function Design

Keywords: *args, **kwargs, functools.wraps, lambda functions; inspect.signature; decorators

Presenter	James Powell james@dutc.io
Date	Friday, December 4, 2020
Time	9:00 AM PST

Materials:

• Overview
• Hands On

---

Data Analysis & Engineering → “pandas is really good, actually”

Seminar (Wed, Nov 18, 2020; 12 PM PST)

Theme: Data Analysis & Engineering

Topic: pandas is really good, actually

Keywords: Python, data analysis, data engineering, numpy, pandas

Presenter	James Powell james@dutc.io
Date	Wednesday, November 18, 2020
Time	12:00 PM PST

Materials:

• Overview
• Hands On

---

Tools and Approaches → “Testing, Property Testing & hypothesis”

Follow-up (Wed, Nov 11, 2020; 12 PM PST)

Theme: Tools and Approaches

Topic: Testing, Property Testing & hypothesis

Keywords: Python, testing, pytest, hypothesis

presenter	James Powell james@dutc.io
date	Wednesday, November 11, 2020
time	12:00 PM PST

Materials:

• tmate
• Code Reviews
  • Session: Adrian Lewis, Cecilia Brea, Prasad Addagarla (regex)
• Case Studies
  • Notes

---

Automation → “Concurrency Approaches & threading, multiprocessing, and asyncio”

Seminar (Wed, Nov 11, 2020; 9 AM PST)

Theme: Automation

Topic: Concurrency Approaches & threading, multiprocessing, and asyncio

Keywords: Python, automation, concurrency, parallelism, threading, multiprocessing, asyncio

Presenter	James Powell james@dutc.io
Date	Wednesday, November 11, 2020
Time	9:00 AM PST

Materials:

• Overview
• Hands On

---

Programming Fundamentals → “Object-Orientation in Python and the Python Object Model”

Seminar (Fri, Nov 6, 2020; 9 AM PST)

Theme: Programming (& Python!) Fundamentals

Topic: Object-Orientation in Python and the Python Object Model

Keywords: Python, object orientation, data model, object model

Presenter	James Powell james@dutc.io
Date	Friday, November 6, 2020
Time	9:00 AM PST

Materials:

• Overview
• Hands On

---

Data Analysis & Engineering → “How Python, numpy, and pandas ‘Fit’ Together”

Follow-up (Mon, Oct 30, 2020; 12 PM PST)

Theme: Data Analysis & Engineering

Topic: How Python, numpy, and pandas ‘Fit’ Together

Keywords: Python, data anaylsis, data engineering, numpy, pandas

Presenter	James Powell james@dutc.io
Date	Monday, October 30, 2020
Time	12:00 PM PST

tmate: https://tmate.io/t/ro-AMpD8zLZnpdHq5pPMaKx3wKGj

Materials:

• Case Studies
  • Notes
  • GroupBy
• Code Reviews
  • Session I: Macos Martinez
  • Session II: Nitin Kuncham
  • Session III: Anubhav Sinha

---

Tools and Approaches → “Testing, Property Testing & hypothesis”

Seminar (Fri, Oct 23, 2020; 9 AM PST)

Theme: Tools and Approaches

Topic: Testing, Property Testing & hypothesis

Keywords: Python, testing, pytest, hypothesis

presenter	James Powell james@dutc.io
date	Friday, October 23, 2020
time	9:00 AM PST

Materials:

• Overview
• Hands On

---

Device Automation → “Automation with asyncssh and paramiko”

Follow-up (Mon, Oct 12, 2020; 12 PM PST)

Theme: Device Automation

Topic: Automation Using asynchssh and paramiko

Keywords: Python, Device Automation, ssh

presenter	James Powell james@dutc.io
date	Monday, October 12, 2020
time	12:00 PM PST

tmate: https://tmate.io/t/ro-kGFKnayD9JjAdHBMDvJH4EFTh

Materials:

• Notes

---

Data Analysis & Engineering → “How Python, numpy, and pandas ‘Fit’ Together”

Seminar (Mon, Oct 12, 2020; 9 AM PST)

Theme: Data Analysis & Engineering

Topic: How Python, numpy, and pandas ‘Fit’ Together

Keywords: Python, data analysis, data engineering, numpy, pandas

presenter	James Powell james@dutc.io
date	Monday, October 12, 2020
time	9:00 AM PST

Materials:

• Overview
  • traces.jsonl
• Hands On

---

Device Automation → “Automation with asyncssh and paramiko”

Seminar (Fri, Oct 9, 2020; 9 AM PST)

Theme: Device Automation

Topic: Automation Using asynchssh and paramiko

Keywords: Python, Device Automation, ssh

presenter	James Powell james@dutc.io
date	Friday, October 9, 2020
time	9:00 AM PST

Materials:

• Overview
• Hands On
  • Debian
    • Dockerfile
    • user.sh
  • Ubuntu
    • Dockerfile
    • user.sh
• Really Hands On
  • example.py

---