Collected sources and patterns will appear here. Add from search, explore, or the patterns library.
TableMetadata, ClusterResources -> List<JobSplit>
Group physical file blocks into optimally-sized input partitions aligned with execution slots and network topology for map-reduce scheduling.
Problem it solves
Arbitrary grouping of data blocks leads to network bottlenecks and unbalanced compute jobs.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.