[jira] [Created] (FLINK-6936) Add multiple targets support for custom partitioner

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Created] (FLINK-6936) Add multiple targets support for custom partitioner

JIRA jira@apache.org
Xingcan Cui created FLINK-6936:

             Summary: Add multiple targets support for custom partitioner
                 Key: FLINK-6936
                 URL: https://issues.apache.org/jira/browse/FLINK-6936
             Project: Flink
          Issue Type: Improvement
          Components: DataStream API
            Reporter: Xingcan Cui
            Assignee: Xingcan Cui
            Priority: Minor

The current user-facing Partitioner only allows returning one target.
public interface Partitioner<K> extends java.io.Serializable, Function {

         * Computes the partition for the given key.
         * @param key The key.
         * @param numPartitions The number of partitions to partition into.
         * @return The partition index.
        int partition(K key, int numPartitions);
Actually, this function should return multiple partitions and this may be a historical legacy.
There could be at least three approaches to solve this.
# Make the `protected DataStream<T> setConnectionType(StreamPartitioner<T> partitioner)` method in DataStream public and that allows users to directly define StreamPartitioner.
# Change the `partition` method in the Partitioner interface to return an int array instead of a single int value.
# Add a new `multicast` method to DataStream and provide a MultiPartitioner interface which returns an int array.

Considering the consistency of API, the 3rd approach seems to be an acceptable choice. [~aljoscha], what do you think?

This message was sent by Atlassian JIRA