Here’s how Netflix uses Python.

Operations

Neftlix’s demand engineering team build resiliency into the network by providing regional failovers and orchestrating the distribution of Netflix’s traffic.

“We are proud to say that our team’s tools are built primarily in Python,” the team writes.

“The ability to drop into a bpython shell and improvise has saved the day more than once.”

Tools used by the team include:

• NumPy and SciPy to perform numerical analysis
• Boto3 to make changes to AWS infrastructure
• rq to run asynchronous workloads
• Flask APIs are used as a wrapper around the orchestration tools above.
• Jupyter Notebooks and nteract are used to analyze operational data and prototype visualization tools. Neflix uses Python to build custom extensions to the Jupyter server that allows engineers to manage tasks like logging, archiving, publishing and cloning notebooks.

Meanwhile, the big data orchestration team provide services and tooling for scheduling and executing ETL (Extract, Transform, Load) of data and adhoc data pipelines.

The team use Jupyter Notebooks with papermill to allow the scheduler to provide templatized job types, for example Spark.

Also used is pygenie, a Netflix-built client that interfaces with Genie, a federated job execution service.

Statistical analysis

Netflix’s CORE team uses many Python statistical and mathematical libraries, also including NumPy, SciPy, ruptures, and Pandas, which help analyse thousands of signals after an alert.

Python has also been used to develop a time series correlation system, as well as a distributed worker system to parallelize large analytic workloads.

On top of that, Python is also typically used for automation tasks, data exploration and cleaning, and visualization.

How Netflix uses Python: Streaming giant reveals its programming language libraries and frameworks