In this talk, I will give you tips and practical advice on what steps to follow and how to plan your data science project to avoid making the most common mistakes during its development.

Despite my limited experience delivering data science projects, I have learned how to avoid certain mistakes. In this talk I will teach you how to prevent them and save you lots of headaches.

learn how to avoid the common mistakes when starting a data science project

In this talk you will learn tips on:

- Defining requirements
- Outlining a data science project
- Reproducibility and Readability checklist
- Best practices for writing Documentation
- Useful python tools
- Tips on presenting your findings
                       
This is a talk is intended for beginner audience.

Sara Iris Garcia

The beginner’s data science project checklist

If you have ever worked on an internal company project, you may feel it deep in your bones.
Let’s say that you discovered a need for a generic technological component in your organization’s tech stack. You identified stakeholders, gathered requirements, and started agile iterations on providing it. Then comes a day when you can show the MVP to your internal client! Yet… the client has lost his interest: maybe right now he says that he has already come up with his own temporary solution and he has no intention to switch to another one?  Building internal products differs from commercial ones - there is no flow of cash and your clients are fully transparent.

In this talk, I would like to share with you my experience and tips connected with developing such internal tools and standards. All of this from the perspective of a member of the Machine Learning Infra team that is delivering its solutions to a rapidly growing ML department in a company whose product is used by 300 million unique users per month.

But let’s be specific! I will talk about:

- Common pitfalls and try to dig up the reasons for why they happen when developing internal solutions
- How one can approach delivering tools (spoiler: pilot programs, guilds, and more!)
- Learnings from introducing such approaches (what worked, what didn’t) in our case

Building internal products differs from building commercial ones. Let’s talk about common pitfalls, ways of addressing them, and how one can excel at making their (internal) clients happy.

Paulina Winkowska

The intricate art of making your (internal) clients happy - the story from a Python-centered Infra team

This talk aims to encourage beginner developers not to underestimate the skills and benefits they can bring to various non-IT environments. I joined a team focused on drought research at the Czech Academy of Science in 2016 with a fresh degree in Geoinformatics and minimal experience with coding. Thanks to this very little initial knowledge, we were able to build a robust system providing drought monitoring and forecast for Czechia and also the whole of central Europe. We were able to fight through text files, user inputs, and geodata of all sorts and say goodbye to manual processing thanks to Python and its geospatial and data processing libraries. On the technical side, the presentation should introduce some of the handy geospatial and data processing tools to get your hands on any task, from producing colorful maps to analyzing time trends in satellite imagery. It should also be a guide on identifying needs and building the most necessary data manipulation processes from scratch.

Python in Czech Drought Monitoring System for geospatial data processing and automatization of tasks in the scientific team.

Monika Bláhová

Czech Drought Monitoring system – a journey from manual work to global drought monitoring and machine learning, powered by Python

In this talk we will look into a few bug cases or doubtful features in CPython some of which are still present (and known to bugs.python.org) and may impose a security risk for admins or organizations.

In this talk you can learn about some security risks comming from known(!) CPython bugs or doubtful features

In this talk we will look into a few bug cases or doubtful features in CPython some of which are still present (and known to bugs.python.org) and may impose a security risk for admins or organizations.

We will learn why running Python interpreter in random directory can be harmful which is related to interpreter libs loading, a possibility for installed modules to inject code into any Python script execution (even if the installed library is not imported), a socket.inet_aton issue that actually comes from glibc and risks involved with those cases and possible mitigations of those risks.

@EDIT After talk:
- Watch the talk on https://youtu.be/tRtxCCRdZOs?t=12251
- Slides are available on https://ujeb.se/pybugs

disconnect3d

CPython bugs & risky features

Are you working with threads, processes, or more generally “workers”? And do you have blocks of code that must not be called concurrently? Maybe you didn't even realise it until your system experienced a bug you could not reproduce until the stars aligned. Then you surely know that hope is not the answer to a robust system, we must be prepared to face worst-case scenarios.

This talk will first briefly present race conditions, a staple in concurrent computing. We will then compare implicit and explicit concurrency management in your core logic, that is whether you delegate or craft protective logic yourself. Next comes testing, the real crux of the talk, where we will demonstrate how to manufacture a race condition. Finally we will explore how to solve such problems with the built-in tools the Python standard library offers.

Get a quick tour of race condition handling in Python: what it is, the built-in tools, and how to test for it.

Anybody working with concurrency (threads, processes, or more abstractly workers) eventually encounters a _race condition_. This kind of hair-pulling problem is a staple in computer science. Sometimes you can offload the problem to a lower layer, usually to the database engine, bundling it with atomic transactions. But there are times when you have to take charge of worst-case concurrency scenarios, when your architecture and constraints do not leave room for implicit concurrency management. Then, you need to carefully craft the paths in and out of the critical section, avoiding deadlocks along the way.

In this talk, we will focus on how to test your program with a manufactured race condition. If you know it can happen, you'd better cover it in your test suite. We will define a server program called in parallel by multiple clients to illustrate the problem at hand. Then, we will manually set-up a race condition during testing to explore broken behaviour and problematic code.

The Python standard library offers all the tools we need to cater for race conditions. We hope the tour we will give allows you to exercise your own race conditions more easily. Why, we even think you will have all the cards in hand to counter them!

Borjan Tchakaloff

Taking charge of your race conditions

IoT devices are increasing in power and capabilities, now allowing developers to deploy machine learning models on the device. This talk will analyse a boxing training session with motion sensors onboard multiple IoT devices using TinyML: a TensorFlow-based framework. Ultimately, these machine-learning powered IoT devices provide feedback to boxers on their technique.

Want to use wearable tech and machine learning to become a boxing champ?

Internet of Things (IoT) devices are becoming more advanced through additional sensors, reduced size and increased computational power. In particular, this increase in computational power allows one to run previously-trained machine learning algorithms natively on an IoT device. 

This presents an exciting opportunity: IoT devices often feature a variety of onboard sensors which can be used as inputs into a machine learning algorithm. 

This talk will use the presenter's boxing training as a practical example of applying sensor data to a machine learning algorithm. In particular, this talk will demonstrate using motion sensor data obtained on an Arduino Nano 33 BLE Sense configured with TensorFlow Lite. This talk will discuss the entire analytical process from problem and data analysis through to algorithm training and deployment. It will also discuss boxing concepts and how these concepts are modelled in an IoT context.

The links to code are provided below:
- https://github.com/ajosephau/boxing_tracker_nano_ble_sense
- https://github.com/ajosephau/boxing_tracker_wio_terminal

The slide deck is available here:
- https://www.dropbox.com/s/7448hf8q9umnnse/Wearable%20Tech%20boxing%20technique.pdf?dl=0

Anthony I. Joseph

Applying machine learning capabilities to wearable IoT devices for boxing technique management

The Blue Brain Project strives to simulate the whole mouse brain. The amount of data and code this implies is astoundingly high, and it requires the development of software tools that have both a strict and clear structure and that are resilient to errors that will manifest when developing complex code. Workflows are a straightforward way to maintain structure in toolchains that grow increasingly complex. Workflow management packages such as Luigi bring functionality to run different tasks in parallel, keep track of completed tasks and improve the reproducibility. This poster will present two Blue Brain Project software tools, the e-model-packages software and BluePyEModel, focusing on the creation and distribution of in-silico neuron cells. The e-model-packages software collects cells from an in-silico brain circuit and arranges them in individual ‘neuron packages’ to be distributed to the public through the Blue Brain online portals. The cells packages it creates are designed to be easily run with the open source EModelRunner package. The BluePyEModel software creates and optimizes in-silico neurons and is able to reproduce features from real neuronal experiment recordings. Under the hood, it uses the open source BluePyEfe and eFel packages to extract the electrophysiological features from experimental cells, and the open source BluePyOpt simulator to optimize and validate the parameters of the in-silico neurons.

On the benefits of using workflows: insights from two software tools in the context of computational neuroscience

Aurélien Jaquier

In the last few years, lots of new database engines have been developed, making the selection process even more challenging than it was before, if you want to maintain an edge.

The talk will give an overview of what to consider in different situations.

The database market is thriving these days. Stay on top by learning what's available and which options to consider in your aplication design.

In the last few years, lots of new database engines have been developed and existing ones have been extended to cover new application spaces and features, making the selection process even more challenging than it was before, if you want to maintain an edge.

The talk will highlight the most important database engines to consider and their strengths when using them with Python applications, covering relational databases for general purpose tasks, data warehouse workloads, data analytics, machine learning, streaming data and massive scalability, to name a few aspects.

Talk slides: https://downloads.egenix.com/python/EuroPython-2022-Talk-Choosing-the-right-database-for-your-next-project.pdf

Marc-Andre Lemburg

Choosing the right database for your next project - Looking at options beyond PostgreSQL and MySQL

Visual Studio Code along with GitHub, Codespaces, and Azure Machine Learning have been investing substantially into tools and platforms to make the lives of Python data scientists easier, and we want to share why VS Code is now the #1 tool for Python Data Scientists according to the 2021 Python Software Foundation Developer Survey, and how you can leverage VS Code to take your data science productivity to the next level.

This talk will walk through several common Python data science scenarios, showcasing all the productive tooling VS Code has to offer along the way. As a sneak peek, we will be demoing a best in class Jupyter Notebooks experience with VS Code Notebooks, a revolutionary new data cleaning / preparation experience with Data Wrangler in VS Code, collaboration features with GitHub and Codespaces, Azure Machine Learning for deployment, and more!

Come learn why VS Code is now the #1 tool for Python Data Scientists according to the 2021 Python Software Foundation Developer Survey, and how you can leverage VS Code to take your data science productivity to the next level.

Jeffrey Mew

Python & Visual Studio Code - Revolutionizing the way you do data science

Ever had issues to manage your python packages and environment? Do you know how to create and share a package to the community? It can be challenging if you've never done it, but it also doesn't have to be hard. There is always a better tool to fit our needs.

In this presentation, I'd like to discuss how Python's package managers appeared and evolved with time. Discussing pip, pipenv, and poetry, presenting each of their weak and strong points. Also intend to present how to package and publish a simple code with each one of them, and suggest which package manager should you choose, whether you are just starting with python, or feel like there is something bothering and never knew you could solve it easily and painless.

Slides can be accessed here:
https://github.com/vinigfer/europython_2022_slides

From pip to poetry - Python (many) ways of packaging and publishing

Vinícius Gubiani Ferreira

Many organizations have migrated their data warehouses to datalake solutions in recent years.
With the convergence of the data warehouse and the data lake, a new data management paradigm has emerged that combines the best of 2 approaches: the botton-up of big data and the top-down of a classic data warehouse.

Data Warehouses Meet Data Lakes, how build a modern data architecture with python

In this talk, I will explain the current challenges of a datalake and how we can approach a 
moderm data architecture with the help of pyspark, hudi, delta.io or iceberg.
We will see how organize data in a data lake to support real-time processing of applications 
and analyzes across all varieties of data sets, structured and unstructured, how provides 
the scale needed to support enterprise-wide digital transformation and creates one unique source of data 
for multiple audiences.

Mauro

Data Warehouses Meet Data Lakes

A huge source of friction in software is publishing new releases. Somebody has to manually review commits and write a change-log, add a version number, and publish to PyPI. We will cover a better way: an automated process in which new versions are automatically published by merging pull requests.

A huge source of friction in software is publishing new releases. Somebody has to review commits, write a change-log, add a version number, and publish to PyPI. There’s a better way: automatically publishing new packages merely by merging pull requests.

Empowering anyone to issue a new release by submitting a pull request? At first glance, it seems like an outlandish idea. Upon further inspection, however, the benefits become clear.

One of the biggest sources of friction in software development is packaging and publishing new releases. Somebody has to sift through the commits and write a change-log, tag it with a new version number, and publish the package to PyPI. And usually only one or two people have the access necessary for this last step.

The unfortunate result is an infrequent release cadence. Bug fixes and new features are sitting there in the `main` branch, but hardly anybody is benefiting from them because they aren’t in a shipped release yet.

Thankfully, there’s a better way: a continuous release process where new versions are auto-published via pull requests — without any manual monkeying.

In this environment, all pull requests must include a release file that describes the changes within. This file must also include whether the new version should be a major, minor, or patch release.

The continuous integration (CI) process looks for this file, and if one hasn’t been included, the tests will fail, and the pull request won’t be merged. Otherwise, a maintainer merges the pull request, and the CI process then uses the release file to put the description into the change-log, increment the version number, commit, tag, build, and publish the new release to PyPI.

So with almost no human input, every code contribution results in a new release in a matter of minutes. Every feature and bug-fix has its own release, without anyone having to remember to package and publish a new version. Plus, if a bug is found, it’s now much easier to trace it to a specific release version.

Perhaps the best part is that all contributors get to issue their own releases. What better way to welcome new contributors than to reward them with a dedicated release composed entirely of their own work?

Attendees of this talk will take home the following knowledge and skills:

* what kinds of problems are introduced by an infrequent release cadence
* how automated releases solve those problems and provide myriad ancillary benefits
* how to format and parse the release file
* how to use continuous integration (CI) systems to automate releases
* how to add deployment code so new versions are released when PRs are merged
* how to use [AutoPub](https://github.com/autopub/autopub) to facilitate this process

Justin Mayer

Python Packaging Automation — Auto-Publish to PyPI via Pull Requests

Using Pandas, Python and Plotly to locate potential trend reversals in Stocks, Crypto or any OHLC feed.  Learn how to locate Fibonacci retrace levels and predict price reversal zones for the lowest risk entry to a trade.

Using Pandas, Python and Plotly to locate potential trend reversals in Stocks, Crypto or any OHLC feed.

The beginner’s data science project checklist

Abstract

Description

The speaker