pip install pyoptimus
In a little more than 10 lines you can, remove white spaces, accents in all columns, lowercase all columns data, drop a "dummyCol", transform date format, sort a column, convert integers to a "string", replace "taco" per "taaaccoo" and "pizza" per "pizzza"
Load and save locally (or remotely) Excel, CSV, JSON, parquet, Avro. Get and insert data from Mysql, Redshift, SQL Server, Postgres, Oracle, Casandra, and Presto.
Think Optimus as a universal way to access many of the dataframe technologies available in python. Optimus can works with Pandas, Dask, Spark, Vaex, cuDF, and Dask-cudf as backend.
Although most dataframe API tries to mimic Pandas there are always little differences in the way these dataframes work. With Optimus, we want to let you code and then use the technology and infrastructure available to you to process your data.
For CPU, Optimus can use Pandas, Dask, Spark, or Vaex. For GPUs, Optimus relies on cuDF and Dask-cuDF.
Optimus focused on give you the best tools for all your data processing needs. From data quality, plotting, parsing dates, URLs, email, and NLP preparation.
Optimus give you the best performance, so you don't have to reinvent the wheel.