7

I'm designing API for a python library that accepts asynchronous input and produces the asynchronous output: various signals come in, and various signals are generated in response (there's no one-to-one relationship between input and output signals).

The input could come from a network socket/file/interactive (terminal-based) user input; or from a web framework view function. Similarly, the output may have to be sent to a socket/file/screen, or returned to a caller (in this last case, it should be buffered, and all output accumulated since the last call should be returned). The choice between different I/O channels is made once at the beginning and isn't changed while the program is running.

I assume the I/O layer uses multiple threads or processes or is asynchronous. And it calls my library functions.

What are the pros/cons of accepting as arguments two stream objects (for input and output); exposing functions that need to be called when an input is available or output is desired; using messages to communicate? Or perhaps some other approach is better?

ratchet freak
  • 25,706
  • 2
  • 62
  • 97
max
  • 1,075
  • 11
  • 19
  • that feel like some kind of IO's god no ? Are you sure you're not aiming too wide ? Whatever, there is some missing information : what if your application crash : does the pending works is lost or do you need to be able to execute it after the application restart ? – Walfrat Nov 17 '16 at 09:58
  • @Walfrat Yeah, now I'm guessing it's just too much to ask, since it's kinda combining synchronous and asynchronous API in one. Re: pending work - that would be too hard, it's fine if it's lost; although some limited recovery can be achieved by some middle layer buffering and resubmitting some inputs when it figures out that they definitely weren't processed. – max Nov 17 '16 at 16:39
  • Well a very generic API would be that you act like everything is aynschronous, and you give an input stream, an output stream, and a callback which get called when the work is done. – Walfrat Nov 17 '16 at 18:31
  • For my feeling the question is to broad. It is hard to answer without knowing details of the usecase. – k3b Mar 21 '18 at 17:38
  • Are you going to validate input? If yes, how and where? – yegodm Mar 22 '18 at 14:05

1 Answers1

2

Briefly:

  • messages: a straightforward paradigm, and often the easiest to interface with, especially if you want to target a wide range of i/o (tcp, http, terminal...). In most cases, you have an independent in-between layer to cache and handle i/o streams.

  • functions: well, that's perfect for libraries used in the same language. But if you want to interface with it at a higher level, you will need a wrapper around it anyway.

  • streams: accepting streams directly in the lib is similar to messages but has a drawback. Mainly, everything regarding what can go wrong with streams (disconnected, timeouts, errors...) tends to end up as exception handling in your "business logic" instead of a separate layer.

IMHO, I wouldn't see it as an "this or that" question. Rather it might be interesting to look at it from this perspective:

streams are used to transmit messages which are the result of function calls

dagnelies
  • 5,415
  • 3
  • 20
  • 26