Ad

Sunday, August 11, 2024

Making everything a connected device

Abstract

Today we are definitively getting to a point where it becomes challenging to draw the line between a household product being bought and built to serve its consumer, and that same product being designed and equipped for serving other entities beyond the knowledge of its official owner.

Many forms of technological progress converged to get us to a point where it becomes ambiguous and unclear what is the full scope of the sophisticated technological products that are all around us and so often extremely affordable, in spite of its immense capabilities. One of the most obvious examples is of course the smartphone. Its computing power often overlaps that of a personal computer, while still fitting in a pocket and having a plethora of sensors and hardware features, some of which the user may never know these are present and for what purpose these serve.

At home, is more and more the normal scenario to have the most prosaic devices, such as light bulbs and switches, all equipped with a chip that renders these connected to the Internet. Once this is done, all bets are off. For the consumer is very convenient, because at the reach of an app, one can control and monitor his property or perform tasks remotely.

Today is a bit of a gamble. Between the comfort and convenience provided by technology in the realm of the home environment, there is also the security risk, privacy and the user being manipulated and targeted by manufacturer campaigns, or its information sold to third parties for similar purposes.

Still, however powerful and discrete these technologies might be, the greatest joy lies in leveraging its power for the own use, and that is, with no strings attached, and the user really running the show.

That is where I like to sit at. It is true that for IoT tech to become so abundant, cheap, and with routes for open use, the fact is that intentionally or not, manufacturers (regardless of their agendas) gave way for the communities to build on their hardware, but ultimately not being dependent of their clouds and platforms. Projects like Tasmota or ESPHome are one such example, where off-the-shelf products like bulbs and switches are often modified/reflashed to run one of these firmwares.

Concept

Here I am presenting a bit of a different approach though. Starting with a very conventional insect zapper with nothing that can be pinged from the Internet (no electronics whatsoever -  a couple of light bulbs, a ballast, two starter switches, and a Cockcroft–Walton multiplier for generating the HV to zap the insects), I wanted to integrate it better in home automation without requiring external devices such as Wifi sockets or such. 

Implementation

And so I went. The first iteration was to put an ESP32 relay board on it. I used the "ESP32_Relay X1" board which I had used in previous projects with good results. This board has the advantage of having an integrated 230 V insulated SMPS, making it very convenient for integrating in mains powered devices:

It was a relatively uneventful assembly. Drilled some holes in the case and attached the board with the help of some standoffs.


Worked just as it was supposed to. Installed Tasmota32-nspanel (yes, this variant, because with the regular build, the relay GPIO will temporarily go high on boot-up because by default the firmware uses this pin and a few other pins for communication with the PSRAM chips - a poor routing choice from the board manufacturer).

But then for me it felt boring to just be able to turn the zapper on and off. What about if I could also detect and count the number of times the zapper would kill insects? That could be fun.

And so I decided to go to the 2nd stage. The ESP32, as some of you might know, has plenty of GPIO ports, and is overall a much more eclectic chip than its predecessor, the ESP8266. One can connect digital signal inputs, as well as DC analog signals to some of these pins, and use the data and events for their projects.

My challenge was to be able to sense the insect getting zapped. My first assumption was that when an insect gets zapped, there is a burst of current flowing across the HV power supply of the zapper. But measuring this HV current could be tricky do to safely and without a lot of precautions. Also I did not want to spend too much time with this project and to risk hardware and especially my health with iterations. So I went for an option that seemed a bit safer and that could also provide results. The idea was to measure the mains AC current on the input side of the HV power supply. 

Unsurprisingly in most products of this kind, this device uses a Cockcroft–Walton multiplier for generating the HV. There is basically a set of diodes and capacitors (9 capacitors and 9 diodes to be more exact) where each pair of diodes and capacitors forms a stage. In this case we have 4.5 stages, which means that the output voltage can reach close to  2 * 4.5 times the input voltage Vpp. In this case it means approximately 2070 Volts.


This curcuit also has three resistors. One or two I presume are for limiting the input current (without it there would be one massive jolt of current for recharging the capacitors), while the other acts as a bleed resistor, to make sure the capacitors are discharged before someone might think of putting the fingers where is not supposed to.

So my idea was to (non-invasively) tap into the phase or neutral lines of this AC input in order to measure the current. My expectation was that during and after a zap there should be a flow of current, therefore making it possible to detect the insect getting zapped. For that effect I grabbed one of these CT boards sold cheaply on Aliexpress:

I did a modification on it however: first these have a capacitor that removes the DC offset from the LM358 opamp output. I didn't want that, because I had to supply a positive signal to the ESP32 ADC input. Secondly, these boards output a voltage that varies with the current signal. In this case the AC signal could be difficult to deal with. So I removed the capacitor, replaced it with a diode, and added a 4.7 uF capacitor in parallel with the output. This way I had a stable signal with no ripple, and more or less proportional to the current. Easier to measure by the ADC.

Connected it to GPIO34, which is the pin behind ADC1 of the ESP32, one of the pins that can be used by Tasmota:


At first, after this modification, I noticed stability issues, i.e. often as I would turn on the zapper, the ESP32 would restart. Because before this modification this behaviour didn't occur, I suspected that it could be related to noise being induced to the 3.3 Volt line due to the mains being rapidly switched. As I was powering the CT board from that rail, and as the wires had to cover some distance, I suspected this could be the cause. That is a particularly sensitive rail, as it also powers the ESP32 microcontroller.

In order to fix it, I added a ferrite bead to the 3.3 Volt rail, and twisted the wires going to the CT board around each other. And, yes instability was gone.

Once the hardware work was completed, it was time to play with the software! My first idea was to use Tasmota rules to detect when the ADC input would measure a value above a certain threshold. I tried this approach but it proved unsuccessful, as the short current pulses would rarely get detected. It is not clear from any of  the project documentation, but apparently rules have evaluation periods that are too long for this scenario.

Without that option, the next best thing I could think of, was to define a driver in Berry (yes, with ESP32, Tasmota opens a whole new dimension of flexibility, by providing a scripting language in the firmware itself, that among many other things, allows the user to code drivers for devices not supported natively by Tasmota), and use its every_50ms callback to put the pulse detection cycle inside.

As such, in a driver we can define code that will get called every 50 ms (that is the maximum you can go). Taking advantage of that, I wrote a small driver that would read the ADC1 pin, and check if the value would be above a certain threshold or not. The threshold I have empirically verified as being a probable discharge and less likely to be noise related. Everytime the threshold is crossed, the driver will publish a message to a specific MQTT topic.

import mqtt
import json
import string

event_threshold = 2110

class ZapperController

def every_50ms()
var sensors = json.load(tasmota.read_sensors())
var zapper_current = sensors['ANALOG']['A1']

# Update the sensor state:
if zapper_current > event_threshold
mqtt.publish("tele/ovalesublime-insect-zapper/EVENT", "{\"event_type\": \"zap\", \"detection\":" + str(zapper_current) + "}")
end
end
end

zapper_controller = ZapperController()

tasmota.add_driver(zapper_controller)

To have a consumer for that event, I configured Home Assistant to subscribe and handle it as a MQTT event:

mqtt:
event:
- name: insect_zapped
state_topic: 'tele/ovalesublime-insect-zapper/EVENT'
event_types:
- "zap"
availability_topic: "tele/ovalesublime-insect-zapper/LWT"
payload_available: 'Online'
payload_not_available: 'Offline'
qos: 1

Testing

For testing (because you don't have flies around when you need them), I picked up a plastic stick, attached a strip of paper to it, and wet the tip. Then by approaching that paper tip from the electrode bars I was able to provoke a discharge the same way an insect would.


And sure enough I was able to produce events:

Conclusions

If you ask if this solution for detecting events is ideal, I would probably say no. First, relying on the 50 ms polling of the Berry driver is not 100% reliable, especially because we are dealing with short lived events. Second, we are capturing a relatively small current that is difficult to discriminate with a current transformer (CT) that is rated for 5 Amps. It is sufficient, but not a strong unambiguous signal. Third, for events we should resort to whatever provisions exist in the microcontroller and the firmware to handle these events as interrupts. This ensures that the processing is fully asynchronous and no events are missed (the processing will forcefully be triggered as a pin state change is detected). My challenge with applying this  approach is that I didn't have enough time to add the hardware for conditioning the signal coming out of the CT, in order to produce a high signal in case of being above the threshold, and a low signal (0V) if below. This could have been implemented by adding a comparator and an adjustment pot. Unfortunately the ESP32 doesn't seem to have a comparator module internally, which would have been very useful for this application.








Wednesday, May 1, 2024

Monitoring and controlling the swimming pool water level


Ever since I became owner of a swimming pool, a whole new set of chores and responsibilites were unlocked. Some are the repeating manual tasks of keeping the pool and the water clean for human use, while others are more technical and infrastructure related tasks that go along with keeping this very special reservoir of water that so much importance acquires in the hot days of summer.

While in a typical swimming pool some basic features are automated (for example controlling th pH, running the filtration or generating the correct amount of chlorine to disinfect the water), there are many others which by default are left to the intervention of a human.


One of the aspects that got me particularly concerned was the maintenance of the correct water level not being an automated process. Especially during the summer, a substantial amount of water is lost every day (at least 5 mm per day - depending on the environmental temperature and humidity). The water level dropping too much is problematic, because as the level approaches the bottom of the skimmer opening, it will cause the pump to ingest air, cavitate and unprime. If this persists for too long the pump will get damaged as its seals and bearings will seize in the absence of water.

With that in mind, I tried to come up with a solution with the following aspects in mind:

  • simple design without many failure modes;
  • be as much as possible based on off-the-shelf and easy to replace components;
  • possible to monitor and remotely control if necessary;
  • integrate with home automation;
  • physically discrete and without any aesthetical impact to the swimming pool and its surroundings;
  • not very expensive to implement and maintain.
While there are different types of solutions that one way or another cover the water level control aspect, these are in general either expensive, limited in terms of integration with other home automation platforms, or impossible to control remotely.

As such I decided to call for my ingenuity, and put together a custom solution that would tick all the boxes.

So basically the solution needed two components: a sensor installed in the swimming pool to detect the low level, and an automatic valve to inject tap water.

The sensor

Because I had good experience with Zigbee devices for their long battery life and ease of integration with home automation platforms, I decided to go for a very particular approach in respect to the sensor: retrofit a Aqara water leak sensor the serve the opposite purpose - detect lack of water instead of presence:


While ideally I would be looking for a proportional indication of the water level, so that for example I could also detect an excessively high water level (which will occur when it rains heavily), I could not readily find a solution that would not imply much more complexity and eventually custom parts.

The simplest approach would be to place this sensor inside the swimming pool, and have it setup so that when it would not detect water, the home automation would kick in and activate a valve. This would be the most basic approach . However it would have some implications: first, even though the Aqara sensor is waterproof, it is likely not designed to be permanently in contact with the water, and especially not water containing a certain concentration of chlorine and other chemicals. Also placing the sensor practically at the water level would seriously degrade the Zigbee signal strength. Lastly, as the water oscillates most of the time, debouncing logic would have to be added to ignore the false positives.  This would also carry the implication that battery life would probably plummet, as the device would have to transmit much more often than normal.

This led me to think about a slightly more sophisticated approach. First, as I did not trust the sensor to be robust enough to tolerate being in contact with the water, I went for designing a custom enclosure where the Aqara sensor could be mounted inside and stay safe and dry all the time.

Because I became accustomed with modelling 3D objects for functional purposes since I bought a 3D printer, it proved to be a no brainer approach to create this enclouse myself and print it.

And so it was. The idea was to use a float switch connected to the Aqara sensor electrodes, so that when the water level would be low, the switch would close and trigger the sensor. For convenience and for keeping it visually discrete, the most logical choice was to put the device inside the skimmer cavity, supported under its cover:


Soon I learned that leaving the float sensor directly exposed to the water surface would lead to a lot of detection events, even while the water level was still substantially high.

This first version was still not based on 3D printed parts but rather on off the shelf parts cobbled together. Soon I found that the plastic enclosure was not a good solution as it would crack and let water in.



That is when I decided to make a 3D printed enclosure from scratch.


This enclosure would solve the excess events issue by measuring the water level not directly from the pool surface, but from the level of water inside a cup that would be part of the bottom of this enclosure:


This cup forms a kind of low pass filter for water waves: as the water is only be allowed in via small orifices, it takes some time for the level to rise and lower, effectively providing a damping behaviour to the variations in the water level as observed in the surface of the swimming pool.



So far, the resin material of the enclosure did not show any signs of wear due to the water or the chemicals present in it, and proved effective at keeping the sensor protected from water ingress. At the top cover right at the end of the threading, I have added a rubber seal, which is essential for keeping the enclosure water tight.

The actuator

With the sensor aspect resolved, it was time to cover the other aspect of the solution: how to automatically inject water into the swimming pool when needed?

Slightly overwhelmed with the idea of messing around with the swimming pool hardware, at first I considered using a modified sprinkler to send a jet of water to the swimming pool directly. But this idea seemed a bit cumbersome and would inevitably make the solution less discrete than desirable.

That is when I realized that it would not be such a big challenge to tap into the PVC pipes initially desiged for the installation of a heat pump:


With these I could effectively inject water into the swimming pool in a very discrete manner. Because the public water supply has a much superior pressure compared to the pool filtering pump (4 + bars compared to less than 1 bar for the latter), there should be positive flow even when the pump is running.

As so it was. I had to study a bit of plumbing in order to determine what fittings I had to buy and how to glue PVC-U piping parts together, but at the end it was a success:


Basically I had to add a 2" pipe  to 3/4" inch reduction fitting, followed by a 3/4" elbow, followed by a retention valve (I wanted to make sure it would not be possible to have water from the swimming pool flowing towards the water supply - however unlikely that would be because of the pressure differences). Lastly the most important bit, the electronic valve that would open the water for injection.

For that I have chosen the GiEX QT06 Zigbee solenoid valve. This valve is normally used for garden irrigation, and it has the interesting feature of having a flow sensor that measures the injected water. Very useful for this application as well.

Before this valve I am also using a motorized ball valve which acts as a safety valve which I can close when not injecting water into the pool or having the irrigation running from the public water supply:


The integration of this valve is a bit more custom and uses a ESP8266 Wifi controller running Tasmota.


Because I am using Home Assistant as the "brains" of the house, and multiple Tasmota and other devices that communicate via MQTT messages, for integrating with these Zigbee devices I resorted to using a Sonoff Zigbee bridge flashed with Tasmota (https://zigbee.blakadder.com/Sonoff_ZBBridge.html) and Zigbee2Tasmota (on the Zigbee coordinator chip).



Then with all the hardware prepared and the Zigbee devices paired to the bridge (very simple procedure, some information can be found here - https://tasmota.github.io/docs/Zigbee/#introduction), this is where all the fun begins. 

Configuring Home Assistant

On HA we need to define the entities that correspond to the Zigbee devices we are adding. In this case the level sensor and the valve.

For the valve I defined the following MQTT switch in the HA configuration.yaml file:

mqtt:
  switch:
    - name: swimming_pool_water_injection
      state_topic: 'tele/zbbridge/SENSOR'
      value_template: >
        {% for key, value in value_json.ZbReceived.items() %}
          {% if 'Name' in value and value.Name == 'valve' %}
            {% if 'WaterState' in value %}
              {{ value.WaterState }}
            {% endif %}
          {% endif %}
        {% endfor %}
      command_topic: 'cmnd/zbbridge/zbsend'
      qos: 1
      payload_on: '{"device":"valve","Write":{"WaterState":1}}'
      payload_off: '{"device":"valve","Write":{"WaterState":0}}'
      state_on: '1'
      state_off: '0'
      availability:
        - topic: "stat/zbbridge/RESULT"
          payload_available: true
          payload_not_available: false
          value_template: "{{ (value_json['ZbStatus3'] | selectattr('Name', 'eq', 'valve') | map(attribute='LastSeen') | first | int) < 14000 }}"
      retain: false

This implies that we first gave the valve a friendly name on the Zigbee bridge, via the ZBName command. For example in the Tasmota console:

zbname 0x2916,valve


Regarding the sensor, I have defined the following MQTT binary_sensor:

mqtt:
  binary_sensor:
    - name: swimming_pool_level_low
      state_topic: "tele/zbbridge/SENSOR"
      value_template: >
        {% for key, value in value_json.ZbReceived.items() %}
          {% if 'Name' in value and value.Name == 'level-sensor' %}
            {% if 'Water' in value %}
              {{ value.Water }}
            {% elif 'ZoneStatusChange' in value  %}
              {{ value.ZoneStatusChange }}
            {% endif %}
          {% endif %}
        {% endfor %}
      device_class: problem
      payload_on: '1'
      payload_off: '0'
      availability:
        - topic: "stat/zbbridge/RESULT"
          payload_available: true
          payload_not_available: false
          value_template: "{{ (value_json['ZbStatus3'] | selectattr('Name', 'eq', 'level-sensor') | map(attribute='LastSeen') | first | int) < 3300 }}"

This exposes the Zigbee level sensor as a binary sensor in Home Assistant, allowing it to be used in monitoring and automations.

For the automation I considered important to add some hysteresis (not trying to compensate the water level on every received event), and alert and give up if it tries to compensate too often:

- id: swimming_pool_water_level_low
  alias: swimming_pool_water_level_low
  trigger:
    - platform: numeric_state
      entity_id: sensor.total_swimming_pool_level_low
      above: 3
  condition:
    condition: and
    conditions:
      - condition: template
        value_template: "{{ (as_timestamp(now()) - as_timestamp(state_attr('automation.swimming_pool_water_level_low', 'last_triggered')) | float) / 3600 >= 24 }}"
      # Data is only valid if it is being updated through the Zigbee gateway. If it is down, we cannot trust the sensors:
      - condition: template
        value_template: "{{ states('sensor.zbbridge_wifi_connect_count') != 'unavailable' }}"
      - condition: numeric_state
        entity_id: sensor.level_sensor_last_seen
        below: 3100
  action:
    - service: script.multi_notification
      data:
        title: "Swimming pool level low!"
        message: "Swimming pool level low! Compensating level."
    # Set the zigbee valve to track the water quantity:
    - service: mqtt.publish
      data:
        topic: "cmnd/zbbridge/zbsend"
        payload: '{"device":"valve","write":{"WaterMode":1}}'
    - delay:
        seconds: 4
    # Set the desired water quantity:
    - service: mqtt.publish
      data:
        topic: "cmnd/zbbridge/zbsend"
        payload: >-
          {% set target_volume =  states('input_number.swimming_pool_water_injection_amount') %}
          {{ {"device":"valve","write":{"IrrigationTarget":target_volume}} | tojson }}
    - delay:
        seconds: 4
    - service: switch.turn_on
      target:
        entity_id: switch.swimming_pool_water_injection
    - service: switch.turn_on
      target:
        entity_id: switch.garden_valve

This automation will only starting injecting water if the level is low for more than 3 hours. Also it will refuse to compensate if the last time it injected water was less than 24 hours ago. Because this Zigbee valve has a flow sensor, we are able to tell it exactly how much water (in 1 liter increments) we want to add to the swimming pool.

Once all of this is setup I can monitor everything from the distance of a web browser and a comfortable chair:



Conclusion

With this solution I was able to monitor the water level, have Home Assistant inject water automatically and have one less thing to worry about especially in the Summer where this solution is most necessary as I am physically away from the site to be able to inject water manually.

In the future I plan to also support the opposite, which is to be able to remove excess water when the level is above the skimmer opening. This is also an important addition because in the most rainy periods of the year water will regularly rise above the skimmer port and render it ineffective at colecting debris. Also there is the risk of water infiltrating behind the liner, causing problems to the adhesion between the latter and the pool walls, ultimately leading to its detachment.

Sunday, January 23, 2022

Rooting the Creality Halot One resin 3D printer

I have finally decided to enter the 3D printing bandwagon. Having room for more hardware have always been the main inhibiting factor for not having done it longer ago. But with a little tidying up of the available space the impossible eventually became a reality. At least that was my reasoning, and the selection of a small resin printer seemed like an interesting choice in that respect, at least in what concerns the space occupied by the device itself.