Cleanlab is data-model and data-framework agnostic, a powerful aspect of its design. It doesn’t matter if you’re running PyTorch, OpenAI, scikit-learn, or Tensorflow; Cleanlab can work with any classifier. It does, however, have specific workflows for common tasks like token classification, multi-labeling, regression, image segmentation and object detection, outlier detection, and so on. It’s worth perusing the example set to see for yourself how the process works and what results you can expect.
Snakemake
Data science workflows are hard to set up, and that’s even harder to do in a consistent, predictable way. Snakemake was created to automate the process, setting up data analysis workflows in ways that ensure everyone gets the same results. Many existing data science projects rely on Snakemake. The more moving parts you have in your data science workflow, the more likely you’ll benefit from automating that workflow with Snakemake.
Snakemake workflows resemble GNU Make workflows—you define the steps of the workflow with rules, which specify what they take in, what they put out, and what commands to execute to accomplish that. Workflow rules can be multithreaded (assuming that gives them any benefit), and configuration data can be piped in from JSON or YAML files. You can also define functions in your workflows to transform data used in rules, and write the actions taken at each step to logs.
