Future self: find this content

Pandas and its competitors

pandas uses Arrows dataframes since version 2.0
polars is written in rust and uses also Arrow dataframes, but also features lazy-evaluation. It can also work on chunks, so in principle you can work on data that is larger than your memory. As it is written in rust, you could also write rust code with it if you like to.
duckdb is also written in rust, but aims to be utilising SQL syntax
Arrow is the base dataframe for newer dataframe systems, but itself is quite limited in what you can do with it, from an analyst's perspective.
modin, ray and dask try to solve problems not only on one computer that can load all the data, but to either load data in chunks to work on data that is larger than your memory or computing that scales to more than one CPU or node. Modin is

Comparison of polars and pandas by Floraian Wilhelm in form of a Jupyter notebook based blog post.

DataFrames 2023 is a blog post that compares the aforementioned DataFrame libraries and shows their connections and specific pros and cons for certain usecases. Also it covers Fugue and Daft with which I have no experience so far.

Visual Pandas Selector is a library to visualise and interact with pandas DataFrames that consist of time series data. I haven't tested it so far, but it is definitively on my todo list.

Future self: find this content

Other articles

SQL learning resources

Other articles

links

social