Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [dataflow]

21 questions
12
votes
3 answers

What is a "lifted representation"?

Just ran across this term here: http://www.codemesh.io/codemesh2014/viktor-klang "We'll demonstrate the Flow API—a lifted representation—as well as a pluggable way of transforming the lifted representation into the execution representation—Flow…
Den
  • 4,827
  • 2
  • 32
  • 48
11
votes
0 answers

Implement FBP / Dataflow programming in VBA?

Intrigued by Flow Based Programming a la J Paul Morrison. I wonder if one can implement FBP in Excel/VBA. I searched quite a bit for a Basic-Lang FBP implementation to no avail. I've looked at some FBP implementation details in Python but I'm not…
klausnrooster
  • 231
  • 1
  • 6
9
votes
6 answers

Do you see an use for "Spreadsheet programming"?

A while ago I stumbled upon the concept of using spreadsheets (I mean cells and formulas not Macro code) as a way to specify programming logic. The idea is: create a spreadsheet with a cleanly defined flow of calculations (which sometimes are…
Paolo Bozzola
  • 344
  • 2
  • 8
6
votes
3 answers

Real-time Dataflow Programming

Background Recently, I have been fascinated about the possibilities of dataflow-based programming approaches - in particular, signal/event-based concepts as realized by functional reactive programming, Boost::Dataflow::Signals, and the like. Sample…
Frank
  • 14,407
  • 3
  • 41
  • 66
5
votes
3 answers

Patterns to allow for versatile data-flow hooks in Javascript

I have an application where I'd like to add ways for external modules/components to hook during the processes and data-flow and perform actions/modify data. For the first requirement—performing actions—I assume the best pattern would be to emit…
Sunyatasattva
  • 440
  • 3
  • 9
5
votes
4 answers

Is it bad practice for a module to contain more information than it needs?

I just wanted to ask for your opinion on a situation that occurs sometimes and which I don't know what would be the most elegant way to solve it. Here it goes: We have module A which reads an entry from a database and sends a request to module B…
Helder AC
  • 295
  • 2
  • 7
4
votes
1 answer

Is undo and redo possible/feasible with two-way data binding (vs uni-directional data flow)?

I'm developing an experimental application for work that requires fully persistent undos and redos. I decided to use react/redux/immutable JS tech stack to solve this problem because I saw it worked out nicely. I'm actually really happy with the…
Rico Kahler
  • 240
  • 2
  • 10
3
votes
0 answers

Spark supports cyclic data flow

I read on http://spark.apache.org/ that Spark supports cyclic data flow. How can data flow be cyclic if the RDDs used by Spark are immutable? I thought the processing done by Spark looks like an acyclic graph where the nodes are RDDs and the edges…
bsky
  • 173
  • 4
3
votes
1 answer

How to efficiently implement this background processing chain?

I am working on an audio software that uses The EchoNest web service to identify and retrieve metadata about audio songs and I would like to have some advice on implementing a background processing chain. (to get the full picture, I'll show you what…
aybe
  • 727
  • 6
  • 16
3
votes
1 answer

How do I transparently cache data in intermediate stages of processing?

I am working with MATLAB on a model reduction algorithm. It is basically a data processing pipeline. ckt = generate_ckt(ckt_properties); freq = generate_fpoints(fconfig); result = freq_dom_sim(ckt,freq); red_ckt = run_PRIMA(ckt, red_order); Each of…
Milind R
  • 155
  • 8
2
votes
1 answer

How (whether to?) include Apache Spark in my Architecture

Brief overview of general data flow The general goal of my system is to allow users to upload many different types of files containing data (PDF, CSV, ZIP, etc.), then index it and perform some basic analysis to make it searchable and to be able to…
foxtrotuniform6969
  • 799
  • 1
  • 7
  • 9
1
vote
0 answers

Where should I persist the side effects of actions?

I have a outlining app that I am writing using unidirectional dataflow and some of the concepts from Redux. When the user has selects items in the outline, and clicks indent, an indent action is fired. This needs to update the state of the app via…
bnieland
  • 89
  • 7
1
vote
2 answers

Application design for step by step filling information data flow

Suppose, my application consists of many components, where users need to fill theirs information, something like this: -Select city `-Select library `-Select book `-... a lot of components like above So, application has determined data flow,…
Yurii N.
  • 341
  • 3
  • 12
1
vote
1 answer

How to decide if object should request or receive data?

Deciding whether an object should request data (by calling a function) or receive data (by having a function called from another object) is probably dependent on the exact situation. So how do I decide how to handle this? If a specific example could…
Simon Verbeke
  • 181
  • 1
  • 9
1
vote
1 answer

Using streams to connect programming languages

I'm trying to run a computation-heavy program that creates an image on a website. Is it possible to compute in C++ and have an output stream that connects to an input stream in Node.js to display an image, or alternatively, stream is to the user and…
user939687
  • 13
  • 2
1
2