0

I am looking for the most efficient solution for transferring data from ~100 individual ESP8266 WiFi modules, that have sensors connected to them, to my Raspberry Pi Pico W.

This will be used for a watering system for an outdoor garden. Each plant will gave its own temperature sensor and water valve.

I would like to constantly retrieve the sensor data from all of the devices, as well as send them information if need be (i.e. watering cycle time). I would like to go wireless as it seems to be the cheapest/easiest option for me, however, I am unsure what would be the right framework for this. I started out with a simple socket connection, but it quickly became a nuisance once all devices needed to make a connection.

I also tested out MQTT, however, the big issue I had was if two sensors sent data at the same time, (which I would like them to be able to do) the Broker may only receive one request. I am looking for a solution that has, possibly, more of a queuing system, such as SMTP.

ocrdu
  • 8,705
  • 21
  • 30
  • 42
drew wood
  • 131
  • 8
  • The 802.11 protocol supports 100+ clients but many AP default configurations won’t allow that many on the same network. Check yours? Spread them out over a few different APs? – winny Oct 31 '22 at 15:20
  • 2
    MQTT is a good choice. Regarding collisions, read about Quality of Service (QoS) levels in MQTT. At levels 1 and 2 the client will resend a message if receipt is not confirmed. One tutorial at this link: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/ – Mark Leavitt Oct 31 '22 at 17:04
  • Why a Raspberry Pi Pico W as the master? Anything is possible these days but setting up a server for 100 clients will be hard on a microcontroller. Why not use a Pi4 and run mosquitto, do your watering logic in Python, etc? – tomnexus Oct 31 '22 at 17:12
  • PI4s are great if you can get them. Element-14 has stock arriving in 11 months. – Jasen Слава Україні Oct 31 '22 at 22:12
  • Vital lesson: don’t have all of your nodes transmit at the same time! Use a pseudorandom algorithm to avoid this. As others have mentioned, a pico w is woefully underpowered for handling 100 wifi connections or even 100 socket connections. Tplink and others have commercial grade access points that are reasonably priced but probably slightly more that a pi4. A pi4 might even have trouble supporting 100 wifi connections as its wifi is not intended to be high performance. MQTT is not an unreasonable choice and should work well with a decent broker - not something on a pico w! – Kartman Nov 01 '22 at 11:51

1 Answers1

2

Consider using a simple UDP protocol. Your sensors might need constant measurement, but it doesn't need to be that fast: perhaps once a minute per plant? Something like temp 12.2 C water 1 reporting their current state.

At one message per sensor per minute you're at about 2 packets/sec, which should be easily management. As it's UDP, there's no connection to maintain.

You could even use straight Syslog protocol (RFC 5424) as it's text based and that means all your sensor data can go straight into a log.

Commands in the other direction could be as simple as water on and water off, or water 60 (seconds) depending what you want. Having an automatic timeout is good (so it doesn't matter when a packet gets missed).

jonathanjo
  • 12,049
  • 3
  • 27
  • 60
  • Udp doesn’t get around the need for a wifi connection and this would be the bottleneck methinks, so the message protocol is secondary. Using MQTT is a no brainer. – Kartman Nov 01 '22 at 11:56
  • 1
    @kartman I did mean UDP over IP over wifi. My understanding was that the problem was the number of simultaneous TCP connections maxing out the TCB limit on the central machine, and UDP would address that perfectly. MQTT is a layer above. It's excellent but normally over TCP so may fall foul of the same problem MQTT-SN would certainly be worth looking into, but I haven't used it myself. – jonathanjo Nov 01 '22 at 12:29
  • Thankyou for your insights, however i feel both these methods will fall short. UDP or MQTT would require me to have some sort of connection between the devices, which is severely limited, especially on an esp8266 (ive seen up to 20 UDP Connections at most) I would have to cycle through connections. – drew wood Dec 05 '22 at 21:27
  • There's not really any such thing as a "connected" UDP link. It most socket libraries, there's a faked "connected" state, which usually just means the IP address and ports are associated with a given socket id. Each slave unit is listening on a port and transmitting on the same on; the central unit ditto. Nothing actually has lots of connections "open". If the Pi Pico thinks it has lots of connections, consider open UDP, transmit, close. – jonathanjo Dec 05 '22 at 21:43