In this talk we will introduce PyArrow and talk bout the transformation that the Arrow format is allowing in the Data Analytics world.
PyArrow provides an in-memory format, a disk format, a network exchange protocol, a dataframe library and a query engine all integrated in a single library. But the Arrow ecosystem doesn't stop there and allows you to work integrating multiple different technologies. It can be a swiss army knife for data engineers and it integrates zero cost with NumPy and Pandas in many cases.
TalkPyData: Data Engineering
Relying on Python as his primary development language for more than 15 years, has always been interested in Python as a Development Platform.
He worked as CTO and team leader of Python teams for the past 10 years and is currently core developer of the TurboGears2 web framework and a contributor to the Apache Arrow project.