Hi Stephan, hi Aljoscha, thank you for your responses! I am afraid that I explained the idea of the algorithm not good Aljoscha's question has a good point. We did not have an implementation val newWorkSet = data.join(nodes.filter{ node => !node.isLeaf() }) Unfortunately the exception is still raised. Ragarding the solution set growing: We are generating new nodes with ID But I am afraid that we did not understood the API to emit the new Can you also please have a look on our step function invokation? initializedTree.iterateWithDelta(data, {_.nodeID}, growTree, maxDepth) with the following variables: Thank you very much for your support! Best regards > Stephan Ewen <[hidden email]> hat am 10. Juni 2014 um 16:31 geschrieben: > > > Hey! > > The way you have written the algorithm strikes me actually as a case for > bulk iterations. Delta iterations are typically cases where the work set > sizes goes down (not strictly up, as here). > > In your specific code, the problem is that you can actually only > join/cogroup with the solution set, and produce deltas for it (deltas are > merged in to the solution set). You do not re-assign the solution set. > > In order to grow the solution set by a set of elements, make a delta where > elements have new keys (do not replace any the current elements). When > adding the delta to what is already there, you get a union, with the > special condition that it is a set union on the keys, and if a key existed > before, it gets replaced by the delta entry with the same key. So the > solution set delta is just the result of the "newWorkSet" variable. No need > to union before. > > For the workset, you may have to rethink the algorithm. The way you wrote > it, the workset will always grow. A delta iteration terminates when the > workset is empty. Try and write your algorithm such that the workset really > contains only the changes that are relevant to the next iteration. If the > changes become zero, the work is done and the iteration terminates. > > Stephan |
Free forum by Nabble | Edit this page |