Skip to content

Data splitter #33

Description

@raghukiran1224

Overview

As a CFP user, I would like to split a dataset (e.g., np array, pandas dataframe) into smaller objects that can then be fed into other nodes/pipeline. This is especially useful when we have compute intensive tasks and would like to parallelize it easily.

Acceptance Criteria

  • Design for splitter, should be simple and intuitive
  • Implementation as an extension to the Node construct
  • Tests

Questions

  • What type of semantics does the splitter node define?

Assumptions

Reference

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Prio1Medium priority, needs attention, order of 2-3 weekscfp-datamodelCodeflare pipelines datamodelgood first issueGood for newcomershelp wantedExtra attention is neededuser-storyUser story

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions