How do I achieve non-linear non-dependent control flow using Promises (in server-side ES6)

Question

Coming over from the Java world, I am having trouble translating a multi-threaded approach to IO to the ES6 Promises concept of aysnc IO. Many of the examples I have seen on promises show a linear flow.

promiseFunction
    .then(functionThatReturnsAnotherPromise)
    .then(functionThatReturnsYetAnotherPromise)
    ...

The examples that show a non-linear flow show functions where I must have all the promises complete before I move on:

Promise.all([functionThatReturnsOnePromise, functionThatReturnsAnotherPromise])
    .then(functionThatUsesBothReturnValues);

What I am trying to do is a tree-based control flow where each branch has no dependency on another branch. Consider this chart showing my control flow:

(1) Make an async REST request to get projects
(2) 5 projects received. For each project received, (a) create a project instance and (b) make an async REST request to get teams of that project
(3-7) x Teams received. For each team received, (a) create a team instance, (b) add the team as a member to its parent project instance, and (c) make a REST call to get all team-members of that team
(8-21) x team-members received. For each team-member, (a) create a team-member instance and (b) add it as a member to its parent team instance.

What is important to note here is that in order for 8-21 to happen, 3-7 don't all have to be done. Basically, what I am trying to achieve here is once 2's response is received, do 3, 4, 5, 6, and 7. As soon as 3 is done (not caring about 4-7), do 8-10.

However, I am not sure which Promise constructs to achieve this tree-like control flow, since 8-21 is not all dependent on 3-7's completion.

getAllProjects()
    .then(function(responseJson) {
        var promises = [];
        for (var index in responseJson.value) {
            var project = new Project(responseJson.value[index]);
            promises.push(getTeamsForProject(project));
        }
        return promises;
    })
    .thenAsEachPromiseCompletes(function(responseJson) {
        var promises = [];
        for (var index in responseJson.value) {
            var team = new Team(responseJson.value[index]);
            promises.push(getTeamMembersForTeam(team));
        }
        return promises;
    })
    .thenAsEachPromiseCompletes(function(responseJson) {
        ...
    });

I think what you're looking for is [Continuations](https://gist.github.com/Benvie/5667557). See also [Why coroutines won't work on the web](http://calculist.org/blog/2011/12/14/why-coroutines-wont-work-on-the-web/). — Robert Harvey, Mar 31 '16 at 16:12
See also [Generators](http://jlongster.com/Javascript-s-Future--Generators). — Robert Harvey, Mar 31 '16 at 16:19
@RobertHarvey I feel like Generators would help here (I shoulda mentioned them in the post) because of the iterative nature of what I am doing, but the code construct of linking Generators and Promises is just not obvious to me. Part of the point of this question is what is the *right* (language's intended way) of doing this, because I can find any number of *poor* (in need of refactoring) ways of writing it. — Michael Plautz, Mar 31 '16 at 16:35

score 3 · Accepted Answer · answered Mar 31 '16 at 16:29

The main problem seems to be where to branch and where to join asynchronous executions. In your sample code you're trying to use a single point to join non-dependent operations. The trick here is to use many branches and joins; say you have an array of projects, and load teams. Then you have a list of promises, and for each you will start a new branch and have a new promise. When chained correctly, things start to clear up and look simpler:

getAllProjects().then(function(projects) {
    return Promise.all(projects.map(function (project) {
        return getTeamsForProject(project).then(function (teams) {
            return Promise.all(teams.map(function (team) {
                project.addTeam(team);
                return getTeamMembers(team).then(function (members) {
                    // ...
                });
            }));
        }).then(function () {
            return project;
        });
    }));
}).then(function (projects) {
    // all done
});

Which is a little messy, but can always be refactored to something cleaner like this:

getAllProjects().then(function(projects) {
    return Promise.all(projects.map(fetchProjectTeams));
}).then(function (projects) {
    // all done
});

function fetchProjectTeams(project) {
    return getTeamsForProject(project).then(function (teams) {
        return addProjectTeams(teams, project);
    }).then(function () {
        return project;
    });
}

function addProjectTeams(teams, project) {
    return Promise.all(teams.map(function (team) {
        project.addTeam(team);
        return fetchTeamMembers(team);
    }));
}

function fetchTeamMembers(team) {
    return getTeamMembers(team).then(function (members) {
        // ...
    });
}

Now in the last example, this complex operation is broken down into multiple comprehensible steps.

I now see how `Promise.all(...)` aggregation applies to my non-linear flow, and your example is a good demonstration of that. Applying to my original post, even though 8-10 don't have to wait for all of 3-7 to be complete, 2 _will not_ be complete until 3-7 are, so there still is dependency, which is why the `Promise.all(...)` still applies to this situation. Flattening it out ended up making a ton more sense, too. — Michael Plautz, Apr 01 '16 at 18:33

How do I achieve non-linear non-dependent control flow using Promises (in server-side ES6)

1 Answers1