How can I discover a local network appliance and connect to its built-in webserver?

Question

This question is vaguely Internet of Things (IoT) related - but it's about the intra-net (no external internet connection in this scenario - we are air-gapped). Let's say I have a new smart household appliance. This smart appliance can connect to my wi-fi and has a built-in HTTP server that you can contact with your browser to make use of the appliance. Full disclosure: I'm the designer of the software for this appliance, and I'm trying to work out the user experience for interacting with it via a web browser.

How can the browser on my phone or laptop discover and connect to this appliance? The question is, what mechanism exists in the browser or operating system to let me discover the address of this device? What URL can I type into the address bar or what feature of the browser or operating system can I use to get started to discover and contact the appliance?

Three ideas:

• The best-so-far solution I have is to install a native application on the phone or laptop that does a UDP multicast to discover the appliance (there would be a UDP multicast listener on the appliance that would answer back with the IP address and port of its local HTTP server. Then the native app would launch the system web browser and navigate to that URL. But I don't like this solution because it requires installing a native app, and the whole point was to just use a browser for the UI or make use of existing operating systems.
• The other thing I was contemplating is making the appliance somehow publish a hostname (I don't know much about DNS or whatever it would be) that would be recognized in the browser (e.g.: http://myappliance-31337.local/ or something like that?) This name would be unique (like a serial number), factory-set, and printed on the case of the appliance (and changeable by the user). I'm not sure if that's possible.
• The last thing I could think of is that the appliance hosts its own special-purpose wi-fi network (I don't know much about wi-fi direct, or whatever technology that would be). I think that any request that the browser makes on this network could be directed to the web-server on the appliance. The appliance could then tell the browser its address on the existing wi-fi network. I think I like this idea the best because I believe this technology exists already (like I said, I think this might be what wi-fi direct does...but I'm not sure.) The answer for my question may very well just be "use wi-fi direct", confirming that this is indeed what that technology is for.

Answer 1

In theory, mDNS can be used for zero-configuration networking:

• The appliance connects normally to the available network and receives an IP address via DHCP.
• The appliance listens to mDNS broadcasts on the local network.
• The user navigates to a page on the some-appliance.local domain. If configured properly, this should cause the user's operating system to try mDNS.
• The appliance receives the broadcast and returns a mDNS response with its IP address.
• The user's browser connects to port 80 on that IP address.

This is exactly your UDP multicast idea, except that the protocol already exists. Unfortunately, there are substantial differences in mDNS support between operating systems – test carefully. Domains in the .local TLD should use mDNS automatically, but most operating systems have mDNS disabled by default (as of 2021).

A further limitation of this approach is that you won't get certificates for .local domains, so you won't get HTTPS. This is a problem as browsers are moving to HTTPS by default and will show scary warning messages for HTTP sites.

So let's discuss alternatives.

An alternative I've seen for consumer routers is that the device has a static IP address (trivial for a router) and that there's a public DNS record that points to this static address. However, this is not appropriate for other kinds of IoT devices.

All of this assumes that there already is a network connection. But for wireless networks, this generally requires credentials such as a WPA2 shared secret. For convenience of setup, many devices therefore create their own WiFi network, without any access protection. This network does not have internet access, but can be used for configuration. However, such unprotected networks are likely to be a security risk as they can allow untrusted users in the vicinity to interact with a device before it can upgrade its firmware. Such an unprotected network should not be the factory default. Measures to improve security could be to protect the network with a key that is printed on the device or to only open the network when requested through physical buttons. WiFi Protected Setup uses exactly such approaches in a standardized manner (e.g. the WPS button). Of course, WPS could also be used to easily connect the device to an existing network.

So considering the above, the most reasonable approach for setup seems to be:

• the device should create its own WiFi network for configuration
• this network should only be used for initial configuration, and should not be enabled afterwards
• the network should be protected by a secret that's printed on the device
• since the device acts as a router on this configuration network, it can give itself a static IP
• you can use a public DNS entry to point to this address in the local configuration network, enabling easy discovery
• the device should run a DHCP server on its WiFI network

Discovering the device in an existing network is substantially more difficult. Creating a custom app (which can then use mDNS or similar protocols internally) is the common approach for consumer devices. Of course, alternatives are available. For example, the device could register it's local IP with some central server, but that is a privacy risk. It is also possible to scan the local network via JavaScript on a web page, but that requires that the device's web server uses an appropriate CORS configuration – and might be prevented by future browser security measures.

Answer 2

There are protocols / combinations of protocols designed for this very scenario. They broadly fall under the category of zero configuration networking aka zeroconf. Some popular implementations are Apple's Bonjour (formerly known as Rendezvous), mDNSResponder (an Open Source implementation by Apple), and Avahi, an Open Source cross-platform implementation compatible with Bonjour. Microsoft's protocol is called UPnP (Universal Plug and Play). There are also many implementations targeted at the IoT space for all kinds of footprints down to the smallest sensor platforms.

There was an effort standardizing zeroconf networking in the early 2000s, but the working group was dissolved without being able to agree on a set of protocols. Nonetheless, many operating systems, devices, and applications implement the drafts and ideas that were the basis of this working group. There has also been renewed interest in this area because of IoT.

It uses a combination of three protocols for fully zero configuration networking, but you can also pick and choose only one or two of those protocols.

• APIPA (Automatic Private IP Assignment) uses a pseudo-random number generator with retries to assign an IP address from the 169.254.0.0/16 network to a device. The protocol guarantees that eventually every device will have an IP address and there are no collisions. The equivalent for IPv6 has been designed in from the get-go: every link automatically always has a link-local IPv6 address, so any two devices can always communicate with each other over IPv6 with no configuration whatsoever required.
• DNS-SD (DNS Service Discovery) uses special DNS entries to describe services. For example, to find a printer in the local network, you just resolve the SRV record _ipp._tcp.local.
• mDNS (Multicast DNS) makes it possible to essentially just blindly send DNS Queries into the network as multicast UDP messages and allow anybody to answer them.

So, DSN-SD allows you to find services using DNS without having to know their location or address. mDNS makes it possible to use DNS without a DNS server or any sort of DNS infrastructure. And APIPA (or link-local addresses for IPv6) make it possible to assign IPs to devices without either having to statically configure them or requiring some form of infrastructure (e.g. a DHCP server).

In your case, you wouldn't actually need DNS-SD, because you are not looking for any device that offers a particular service (i.e. "show me a list of all coffee machines"), you are looking for the device itself ("I want the coffee machine with serial number NCC-1701").

mDNS is essentially a combination of your first two ideas.

Answer 3

You are probably using DHCP to get an IP address (if you aren't, you should), and only fall back to a preconfigured address, or an APIPA (169.254.0.0/16) address, if you don't get any DHCP responses after connecting to the wifi for a while.

The DCHP option 12, "Hostname string", defined in RFC2132, allows the client to set a preferred hostname. Initially, the direction of this option was intended to be from server to client, but it has, for a long time, been used by clients to give a preferred name to the server as well. The IETF describes this in https://datatracker.ietf.org/doc/html/draft-ietf-dhc-host-option-considerations-02.txt.

There is also an option 81, "Client FQDN", explained in https://datatracker.ietf.org/doc/html/draft-ietf-dhc-fqdn-option-05.txt. While the option is called "FQDN", the client may send a partial hostname as well, if it doesn't know its full FQDN.

There are many consumer routers that implement one or both of those options, and I've seen the mechanism used in many organizations that allows BYOD devices as well - these organizations typically use a "real" domain name, no .local, in their names. So, in addition to the mDNS method explained in other answers, it probably makes sense to set these two options in your initial DHCP requests (use a unique name, as stated in your question). This won't work in all environments (you need your DHCP server to support you), but help you get a DNS entry published wherever the local domain suffix isn't .local.

Answer 4

As mentioned by others, mDNS possibly offers what you want.

Note that, depending on routers/firewalls/clients, mDNS might not work reliably, or even at all.

We use it for our IoT project (a CO2 sensor + LED display on ESP8266 or ESP32), but for redundancy:

• the micro-controller displays the IP address via USB Serial.
• if desired, it can create its own Access Point, thanks to IotWebConf library.
• it can also connect to a MQTT broker, and send data and its IP there. Some of our users use a Raspberry PI on the intranet as MQTT broker. It's possible to send commands in both direction via MQTT, so we use it to ask "Give me your local IP".
• in theory, it could listen to ESP32 via bluetooth, and use a Bluetooth Serial interface.
• for debugging, it's also possible to simply scan the local network, e.g. with nmap -sP 192.168.123.0/24.

Finally, some of our users have very strict rules at work, and have to completely disable WiFi on their devices. They can still use LoRaWAN to send data, though. They can only send a few bytes every 5 minutes, but in our case, 3 bytes are enough to send CO2 concentration level + temperature + humidity. Clients then connect to https://www.thethingsnetwork.org/ in order to see the data, instead of connecting directly to the IoT device.

Answer 5

Sorry if this is too much of an anectdote, but I recently bought a small wifi print server and its initial configuration worked like this:

• After pushing a button (which in this case was also the reset button, but it doesn't have to be) it started a Wifi Access Point with an SSID printed on the device.
• You would connect to it with a password also printed on the device.
• If I remember correctly, it even triggered the "this Wifi requires sign-in" functionality.
• You would enter the Wifi credentials, IP address, password, etc. that you want to set up.
• After a few minutes the extra Wifi would turn off again.

This was the smoothest experience I had so far with a device setup. The only annoying point was that you of course had to leave your Wifi to connect to the special Wifi. Also, the initial password is hopefully random, but probably not (only diversified) to simplify production.