2

I have been asked to come up with a design for streaming data to the cloud. I have a closed system on a local server. I need to stream recently registered customers or updated customer records on the local DB to a cloud-based CRM and make data available to serve the firm's website. I came up with following design flow:

  • executable to access the database on the local server and post records to the cloud server. This runs every hour. (1)
  • A cloud server on which an api is hosted (2). A client (exe) calls the CRM Api to post records (3)
  • Website gets(4) via api call.
  • Data for website viewing is different from the data posted to CRM

Any suggestions or feedback/criticisms on the design with regards to connectivity etc.. is there a better way to streaming data from closed systems.enter image description here

Mo Ali
  • 21
  • 2
  • The question whether a 1 hour lag is acceptable isn't an engineering question, it's a product question. It's fine if it meets the product requirements. – Avner Shahar-Kashtan Oct 31 '17 at 06:19
  • 1
    Also, what's keeping you from running the synchronization more often if you feel it's necessary? – doubleYou Oct 31 '17 at 06:54
  • @AvnerShahar-Kashtan It is a requirement part of the engineering solution. – Mo Ali Oct 31 '17 at 11:00
  • any feedback on the design, any criticisms are welcome – Mo Ali Oct 31 '17 at 11:01
  • Yes, and so it's not an answer you can get from other people not familiar with the project and its requirements. If this is data that changes every other day, a 1-hour lag is meaningless. If the users check the web site every week, it's meaningless. But we don't know that. – Avner Shahar-Kashtan Oct 31 '17 at 11:01
  • @AvnerShahar-Kashtan - accepted. any feedback to the design? I have edited my original question. – Mo Ali Oct 31 '17 at 11:03

1 Answers1

1

It looks fine, but to evaluate it properly I would want to know more about the details of the problem you are trying to solve instead of just the engineering solution that seems to have been proposed.

Why do you need to support entering data in to a local database just to push it out to the cloud based CRM (SFDC?).

Can you not just get people to enter their data in to that directly?

If it is a popular cloud based CRM there are existing products that will keep your local database and the cloud based CRM in sync so you wouldn't need to write any custom code (SFDC data model is quite complex to code around yourself).

Does the website already exist and you are extending it? The cloud based CRM presumably has a web front end people could use.

Encaitar
  • 3,053
  • 19
  • 26
  • Kudos to you for saying that there are existing products for local db and crm sync. I would like to know more please, what are these products. – Mo Ali Oct 31 '17 at 13:14
  • "Why do you need to support entering data in to a local database just to push it out to the cloud based CRM (SFDC?)." - because I need to persist the data to serve the exisitng website's viewing area for members only. I will be building it. Yes, the CRM has a frontend for users – Mo Ali Oct 31 '17 at 13:22