3

I'm developing a client library. I'd like to provide both Sync and Async interfaces to endpoints on a server. They would be rather easy to implement as completely separate entities, but I would like to follow DRY principles and other best practices. How could one implement such clients, and are there any established patterns of achieving this behavior? Endpoints need to do three things: pre-process arguments to request attributes, send the request and post-process request contents.

An example of both implementations:

def get_resource(id, parameter):
    """
    Docstring with explanations and parameter descriptions.
    """
    params = pre_process(id, parameter)
    response = sync_request(params)
    return post_process(response)

async def get_resource(id, parameter):
    """
    Docstring.
    """
    params = pre_process(id, parameter)
    response = await async_request(params)
    return post_process(response)

This is of course a simplification. Pre-processing often involves logic which I wouldn't want to duplicate, along with the call signature and docstring. So there could be a number of solutions:

Duplicate everything

I don't consider this a good option. It would require maintaining two pieces of the same logic.

Refactor logic into functions

This would mean that pre_process and post_process would quite literally be functions, but the overall call would be implemented twice in two classes or modules.

Refactor into one function

Suppose we had a way of knowing whether we are in async or sync mode. Synchronous functions can then return "early" to provide async functionality through a truly asynchronous function that duplicates the latter part of the call.

def get_resource(id, parameter):
    """
    Docstring with explanations and parameter descriptions.
    """
    params = pre_process(id, parameter)

    if async_mode:
        return async_get_resource(params)
    else:
        response = sync_request(params)
        return post_process(response)

async def async_get_resource(params):
    response = await async_request(params)
    return post_process(response)

I don't know if it is an advantage that the same function can now be both sync and async. It could even be confusing to use. Depends on the mechanism to provide async_mode I guess.

Refactor into one function with a decorator

This is taking it a bit far, but could work. We can decorate a synchronous function that returns the request parameters. In that decorator we can then decide whether async should be used. If so, return an awaitable, if not, use synchronous calls. A callable function for post-processing the request would be passed into the decorator.

def decorate_call(post_process: callable):
    def decorator(function):
        async def async_send(params):
            response = await async_request(params)
            return post_process(response)

        def wrapper(*args, **kwargs):
            params = function(*args, **kwargs)
            if async_mode:
                return async_send(params)
            else:
                response = sync_request(params)
                return post_process(response)

        return wrapper
    return decorator

@decorate_call(post_process)
def get_resource(id, parameter):
    """
    Docstring with explanations and parameter descriptions.
    """
    return pre_process(id, parameter)

Which of these would be the best solution? Or are there some other, more appropriate methods?

Felix
  • 357
  • 2
  • 14
  • I would much appreciate a comment on the downvote. Is this question not appropriate for SE? Perhaps it could be posted somewhere else. But it is not really a Code Review question either, definitely not Stack Overflow. – Felix Jan 18 '20 at 16:05
  • why do you want to provide a non async wrapper for an async method? – Ewan Jan 18 '20 at 16:05
  • @Ewan What do you mean? To my knowledge, async awaitables can be returned from synchronous functions. I simply want to implement both async and sync interfaces with minimal effort and maintenance. – Felix Jan 18 '20 at 16:07
  • @Ewan Makes total sense if you want to call the method from a background thread. Synchronous calls are easier to handle.Of course the reason for "why do you want to provide X" is often "because customers demand X". – gnasher729 Jan 18 '20 at 16:08
  • I mean the underlying function is either async or not, the calling code gets to decide if it wants to block until the result is returned or not. Wrapping an async within your library is just a bit pointless – Ewan Jan 18 '20 at 16:12
  • @Ewan I am a bit inexperienced with async, so correct if I'm wrong. When e.g. making more than one call to a server, is it not much faster to use async and gather the results rather than calling one by one? Maybe I don't just understand your point. `sync_request` and `async_request` in the examples above are of course blocking and non-blocking respectively. – Felix Jan 18 '20 at 16:15
  • if you are making a request to a server the network round trip takes a long time. your code can either block the thread while it waits or not. async just exposes that choice to the calling code. Hiding that choice, doing the blocking for the calling code only makes sense if the person calling the method doesnt know how to block and wait# – Ewan Jan 18 '20 at 16:59
  • What Evan is saying is that it's not a good idea to provide synchronous wrappers for async functions because their async nature is not something client code can reasonably ignore (not a good idea to abstract the async away). There's limited/questionable benefit to doing it the other way around as well, so this leaves you with duplication, and again, that introduces problems but doesn't give you much in return. – Filip Milovanović Jan 18 '20 at 18:46
  • @FilipMilovanović I think I'm closer to understanding. So given that I want to provide both sync and async alternatives, are you saying that even if refactoring them behind one function could be done like above, it's not advisable because it leads to confusion? I mean, if `async_mode` is false, then the functions are just regular synchronous functions, right? It is for use cases where asynchronous calls are beneficial that I want to provide the async alternative - returning awaitables instead of response contents directly. – Felix Jan 18 '20 at 22:08
  • I think the abstraction you're looking for is the monad. As concrete example of abstracting over sync/async, you could check out ZIO from Scala. – Theodoros Chatzigiannakis Jan 19 '20 at 09:11
  • @TheodorosChatzigiannakis That's really interesting, I haven't heard of that before! Thank you, I'll look into it. – Felix Jan 19 '20 at 09:59

1 Answers1

1

Why do we use DRY? Because if we have the same code in two places, in totally unrelated files, and we change one, we will forget to change the other one and may have very hard to find bug.

Your first approach is nice and simple but repeats itself. But the repetition is a very minimal problem, because it is in two places just ten lines apart. If you need to make a change in one place, it is very easy to make it in the other place.

So approach 1 is KISS, and it is DRY enough.

gnasher729
  • 42,090
  • 4
  • 59
  • 119
  • 1
    Fair enough for that dummy example. But given a more complex client and set of modules it would require the functions or methods to be in similar places. Which is a reasonable way to go. So in your opinion not even the second suggestion of refactoring logic into functions? – Felix Jan 18 '20 at 16:10