Topic: Efficiently accepting data from thousands of sockets...

[JPPoulin]

Hello Apache httpd gurus

I'm contemplating building a complex project around Apache httpd 2.4 and would greatly appreciate your input.

This new system would need to form a 'bridge' between 1) thousands of embedded devices privately uploading data to httpd via sockets and 2) httpd publicly serving static web pages to web browsers who then securely access the above-mentioned embedded data through web sockets.

On the private (embedded side) the use case is as follows:
- An internet-of-things embedded device opens a private socket to httpd.
- httpd accepts a few hundred bytes of embedded data for a few milliseconds and caches it in a RAM cache and a backup database.
- The embedded device disconnects its socket, goes to sleep for a few minutes, and the cycle is repeated indefinitely.  (tens of thousands of devices would do this at regular intervals)

Q1: For this side of the design I need a top-efficiency way to open a public socket to embedded devices and process hundreds of simultaneous data uploads from them at a time. Should the above be done by patching the httpd codebase and insert my own socket code and threading pool or are external modules powerful / efficient / flexible enough to do what I need?

On the public web-facing side the use case is as follows:
- End-users connect to httpd via a URL that contains an embedded connection ID and an authentication hash.
- httpd authenticates the user via the hash and returns a collection of static web pages.
- The client-side javascript code executes and forms a web socket connection to httpd 
- httpd returns the user-requested data to its specified embedded device via web sockets.  (Hundreds of users need to be handled simultaneously)

Q2: I have yet to find a way to efficiently process web socket data in C/C++.  Could you point me to a recommended web socket server or solution that works well with httpd (possibly using mod_proxy_wstunnel?)

Thank you so much for taking the time to share your experience with me!

   Jean-Pierre

P.S. All httpd-related code needs to be in C / C++ and performance / efficiency is important.

[mraddi]

Hello Jean-Pierre,

from your description I guess that the web-based clients (aka "browser") want to see the IoT-data in realtime (therefore the websocket-connection)?
But why use websockets for the connections with IoT-devices? I would suggest using a messagebroker and MQTT as protocol between IoT-devices and the messagebroker because of the protocol's simplicity and it is there for real-time-communication between IoT-devices.
If you need to process the IoT-data further on the webserver you can write some code also connecting to the messagebroker, subscribing to all topics and storing them into a database (or do whatever is needed).
On the frontend-side to the browsers most of the messagebrokers can speak websocket.
I would go for a Docker-based approach with traefik (= container 1) as loadbalancer also handling the HTTPS-encryption, sending the Browser's websocket-requests to the messagebroker (= container 2) and the HTTP-requests to a normal webserver (= container 3).
Have done this for a small application in the past, but not as a cluster - which might be a good idea based on the numbers (and therefore the importance of a 99,9...%-uptime) mentioned by you.

Best regards
Matthias

[JPPoulin]

Hello Matthias,

Thank you very much for taking the time to share this useful suggestion.

I agree that a MQTT-based solution to a message broker to web sockets would make a lot of sense... There are MQTT-based samples on the CC3235 wifi processor my product is based on and I will investigate them further today to find out if that can be a viable path.

Given that I'm not experienced with that technology tree my first instinct was to 'start with what I know' and basically try to cook my own solution so it scales as much as possible...

The general idea is to enable a given server to be as efficient as possible to hopefully have a one-server solution for all the needs... hence keeping things all in one process (adding code to httpd) and have both the embedded side and the public websocket side all running inside the httpd process to streamline everything as much as possible.

Still your suggestion has a lot of merit and I'm going to try to see if your docker-based solution with the recommended packages can run efficiently as my 'first instinct'

The project is still early stage and I very much appreciate your input!

Q1: What C or C++ based websocket solution with source would you recommend that would play nice with httpd and mod_proxy_wstunnel?)

Q2: On the embedded side, is there a module or a package or helpful code that would facilitate me creating the socket to IoT devices (and their attached thread pools?)

Thanks again friend!

Jean-Pierre

[mraddi]

Hello Jean-Pierre,

being a bit late with my response - sorry for the delay.

Q1
Up to now I have never created a module for Apache as there either was a module available matching my needs or it could be accomplished using other programs (a message-broker in this case). So I'm sorry to not have a real answer for this question.

Q2
I'm not familiar with the µC you are using.
But I've used the following module for MQTT-messages (both subscribing and publishing) with the ESP8266
https://www.arduino.cc/reference/en/libraries/pubsubclient/
According to this URL it should also be compatible with TI C3000 !? which at least sounds similar to your µC.

At first sight it might be faster having all functionality in one big block of code.
But on the other side having functionalities split up into different specialized programs makes the whole thing easier to maintain (=> microservice-architecture) and often easier to understand, to troubleshoot and sometimes faster, too. At least that's my experience so far.

Best regards
Matthias

[JPPoulin]

Thanks very much Mattias for taking the time.

Researching this much more, I'll be following your suggestion with MQTT as the CC3235 MQTT code looks solid and I'm able to see data on the server-side as well... I'd be nuts trying to re-engineer all this stuff when it already works well!

Interesting on your embedded MQTT usage... I'll review this work shortly.

Thanks again and best of luck in your projects!