Page 1 of 1

Home automation with Domoticz, ESP8266 and BME280

PostPosted: Mon Dec 26, 2016 11:01 am
by FipS
I've been thinking about how to approach home automation for some time. I wanted something simple, open and easy to deploy with low maintenance cost. I was even considering developing my own custom solution but then came across the Domoticz home automation system, which turned out to satisfy pretty much all my needs and even brings some interesting unseen possibilities that might come in handy in the future. All this served in a nicely polished package, open source and free of charge, nice!

Image

What's essential for me is that Domoticz can be easily installed on a Raspberry PI and that it supports the concept of virtual devices (sensors, switches, etc.) that can be queried and controlled via simple HTTP requests. This gives one great freedom and mainly enables me to use cheap and widely available ESP8266-based nodes as Domoticz clients, these can be deployed around my house in huge quantities serving all kinds of purposes.

I won't cover how to install Domoticz on a Raspberry PI in much detail, but it's very simple and straightforward (nicely covered here), basically all you need is to ssh into your PI and trigger the command bellow:

Code: Select all
sudo curl -L install.domoticz.com | bash

Then it's time to create a virtual device. In this case, we will populate the device with temperature, humidity and barometric pressure data coming from a BME280 sensor connected to an ESP8266 development board via I2C. The actual hardware looks like this:

Image

The image blow shows how to proceed, first we need to add a new virtual hardware. To do this, open Domoticz in your web browser, go to "Setup/Hardware" and name the new virtual hardware e.g. "esp_virtual" (1), then choose "Dummy" (2) as its type, and finally click "Add" (3).

Image

After the new hardware has appeared in the above list, click "Create Virtual Sensors" (4), then give your virtual device a name e.g. "ESP Storage" (5). For the BME280 sensor, choose the "Temp+Hum+Baro" (6) sensor type, this specifies the type and URL format of the data that will be pushed into the virtual device via HTTP requests.

Image

After that, you can navigate to "Setup/Devices" to see the newly created virtual device as shows the image below. Please note that each of the devices has its own unique index "Idx", we will need this information later when issuing HTTP requests from the Lua script that drives the ESP8266.

Image

At this point, it is very simple to verify that the virtual device has been set up properly. All you need is to slightly modify the URL below, namely change the IP address to match your Raspberry PI running Domoticz (10.0.1.20 in my case) and the index of the virtual device you've just added (7 in my case). Then you can open your web browser and enter the resulting URL, a modified version of this:

Code: Select all
http://10.0.1.20:8080/json.htm?type=command&param=udevice&idx=7&nvalue=0&svalue=15.0;59.0;0;1006.0;0

If your web browser prints out the below output in response, it's a good sign that everything works as expected.

Code: Select all
{
   "status" : "OK",
   "title" : "Update Device"
}

If you've already pinned up the virtual device in the dashboard, Domoticz should immediately reflect the values (temperature=15, humidity=59, barometric pressure=1006) that you've just sent encoded in the URL.

Image

The final step is to program the ESP8266 so that it periodically fetches data from the BME280 connected via I2C and send it over to Domoticz. For this, I have prepared a Lua script (for NodeMCU) that you might want to reuse:

Code: Select all
--[ESPDOMOCL_1 BY FIPS @ 4FIPS.COM, (c) 2016 FILIP STOKLAS, MIT-LICENSED]
--[NODEMCU/ESP8266-BASED DOMOTICZ CLIENT, URL: http://forums.4fips.com/viewtopic.php?f=3&t=6898]

--[[
Hardware: ESP8266 development board, BME280 over I2C
Firmware via nodemcu-build.com: nodemcu-master-6-modules-2016-12-18-20-22-54-integer.bin
Required modules: bme280, (file), http, tmr, (uart), wifi
--]]

config = {
  ver = "fips_espdomocl_1, Dec 2016, www.4FipS.com",
  timer = { connect = 1, refresh = 2 },
  -- The below ones need to be customized: 'ssid, 'pwd', 'host' and 'sensor_idx'
  ap = { ssid = "MY_SSID", pwd = "MY_PASSWORD" },
  domoticz = { host="10.0.1.20:8080", sensor_idx = 7, refresh_rate=60 },
  pin = { sda = 1, scl = 2 },
}

function connect(on_ok)
  print("CONNECT: "..config.ap.ssid)
  wifi.setmode(wifi.STATION)
  wifi.sta.config(config.ap.ssid, config.ap.pwd)
  wifi.sta.connect()
  tmr.alarm(config.timer.connect, 1000, 1, function()
    ip = wifi.sta.getip()
    if ip then
      tmr.stop(config.timer.connect)
      print("Connected: "..ip)
      if on_ok then on_ok() end
    else
      print("Connecting...")
    end
  end)
end

function loop()
  refresh()
  tmr.alarm(config.timer.refresh, config.domoticz.refresh_rate * 1000, 1, function()
    refresh()
  end)
end

function refresh()
  bme280.startreadout(0, function ()
    temp10x, _ = bme280.temp() / 10
    humi10x, _ = bme280.humi() / 100
    baro10x, _ = bme280.baro() / 100
    t, h, b = (temp10x/10).."."..(temp10x%10), (humi10x/10).."."..(humi10x%10), (baro10x/10).."."..(baro10x%10)
    print(string.format("BME280: T=%s C, H=%s%%, P=%s hPa", t, h, b))
    url = "http://%s/json.htm?type=command&param=udevice&idx=%d&nvalue=0&svalue=%s;%s;0;%s;0"
    req = string.format(url, config.domoticz.host, config.domoticz.sensor_idx, t, h, b)
    print("REQ: "..req)
    http.get(req, nil, function(code, data)
      if code < 0 then
        print("HTTP ERROR: "..code)
      else
        print("RES: "..code.." "..data)
      end
    end)
  end)
end

print(config.ver)
bme280.init(config.pin.sda, config.pin.scl, nil, nil, nil, 0)
connect(loop)

While connected to the ESP8266 and running the above script you can confirm it works correctly via observing the serial output, it is supposed to look something like this:

Code: Select all
fips_espdomocl_1, Dec 2016, www.4FipS.com
CONNECT: MY_SSID
Connecting...
Connecting...
Connected: 10.0.1.13
BME280: T=24.1 C, H=34.5%, P=1006.8 hPa
REQ: http://10.0.1.20:8080/json.htm?type=command&param=udevice&idx=7&nvalue=0&svalue=24.1;34.5;0;1006.8;0
RES: 200 {
   "status" : "OK",
   "title" : "Update Device"
}

BME280: T=24.2 C, H=41.0%, P=1006.8 hPa
REQ: http://10.0.1.20:8080/json.htm?type=command&param=udevice&idx=7&nvalue=0&svalue=24.2;41.0;0;1006.8;0
RES: 200 {
   "status" : "OK",
   "title" : "Update Device"
}
...

After running the script for several hours, Domoticz will reward you with a nice interactive graph:

Image

I've been running this setup for several days now with two ESP8266 nodes and everything seems to be rock solid, it even survived a power outage without any trouble (all nodes automatically booted up and resumed once power was restored). Overall, I'm pretty happy with this setup and will definitely invest more into this infrastructure in the near future, stay tuned!

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Mon Mar 27, 2017 1:16 pm
by Pino
Hi FipS,

nice project. I want to replicate it myself and I managed to create the sensors in Domoticz. I can update the sensor data by hand with the URL.
After a long wait I have received a BME280 sensor and an ESP8266 ESP12 development board from China. https://nl.aliexpress.com/item/NodeMcu-Lua-WIFI-Internet-of-Things-development-board-based-ESP8266-module/32638137121.html?spm=2114.13010608.0.0.D6wG2m

Can you explain how to connect the BME280 to the ESP8266? I found the ground and the 5 volt on the module but I am not sure which pins to use for the I2C SCL and SDA.
Also it is not clear for me how to load the NodeMCU firmware and the Lua script. I tried the ESPlorer but with no succes.
I hope you can help.
Regards,
Pino

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Mon Mar 27, 2017 10:18 pm
by FipS
Hi Pino,

It looks like you are using the same NodeMCU module like me, connect it to BME280 like this:

Code: Select all
NodeMCU     BME280
------------------
  D1 ....... SDA
  D2 ....... SCL
  3V3 ...... VCC
  GND ...... GND

Regarding the NodeMCU firmware, I use the nodemcu-build.com service that lets me build a custom firmware. Then I use NodeMCU Flasher to flash the firmware into the NodeMCU module. Finally, take a look at ESP8266 Lua Loader, it allows to upload Lua scripts into the module.

I can't go into much detail right now, but let me know if you need more help,
FipS

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Thu Mar 30, 2017 1:54 pm
by Pino
Hi FipS,

thanks for your help. I now know how to connect the sensor. But I am overwhelmed with the choices you can make at nodemcu-build.com. Can I use the default value and I guess that I have to tick the boxes I2C and BME280? With NodeMCU Flasher I have some experience when I build the QuinLED prints. (http://blog.quindorian.org/2016/07/esp8266-lighting-revisit-and-history-of-quinled.html/)
Thanks for your help and regards,
Pino

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Thu Mar 30, 2017 7:34 pm
by FipS
Hi Pino,

For the above Lua script select these 6 modules:
bme280, file, http, tmr, uart, wifi

After the build is complete you will get something like this:
nodemcu-master-6-modules-2016-12-18-20-22-54-integer.bin

When flashing, it is a good idea to also flash esp_init_data_default.bin (check out this for more information):

Image

Here, you can download the exact files I'm using in this project:


FipS

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Fri Mar 31, 2017 12:56 pm
by Pino
Thanks, this helps. I m feeling that I am almost there.
I have successful flashed with ESP8266Flasher.exe the ESP8266:

in file ESP8266MAC.txt you see this:
20170331135810
AP MAC:1A-FE-34-D1-78-06
STA MAC:18-FE-34-D1-78-06


So that looks ok. The blinking LED stops. (this would be the default program from the factory I guess)
With LUAloader (you need to start it with administrator rights in Windows 10) I managed to upload the LUA script. (your script placed in txt file and renamed to init.lua)
Set baud rate manual to 9600.

After restart I see this;
    Communications Error - Check baud rate

     €
    Communications Error - Check baud rate

    nþy%Ýüò©ÒÎ#ÿÈøÿP(ÍY8\úM*‘¨Í_^¼ÿº_¯×mÕÝÕý½¿¯½·¿¿¯¿ÿ bŠbŠJj
    >
    Baud rate changed to 9600

    = node.heap()
    39896
    > Connecting...
    Connecting...
    Connecting...
    Connecting...
    Connected: 10.0.0.171
    PANIC: unprotected error in call to Lua API (init.lua:45: attempt to perform arithmetic on a nil value)

    Communications Error - Check baud rate

    Ø• ÍÌü
    Communications Error - Check baud rate

    ªÈéôÊøÿe g¹
    Communications Error - Check baud rate

    2DgLJ¼Eµºàß›Ýýÿííí
    WARNING! Comm port is disconnected

So it looks that there is something in line 45?
Line45: temp10x, _ = bme280.temp() / 10

What could be wrong?

Code: Select all
    --[ESPDOMOCL_1 BY FIPS @ 4FIPS.COM, (c) 2016 FILIP STOKLAS, MIT-LICENSED]
    --[NODEMCU/ESP8266-BASED DOMOTICZ CLIENT, URL: http://forums.4fips.com/viewtopic.php?f=3&t=6898]

    --[[
    Hardware: ESP8266 development board, BME280 over I2C
    Firmware via nodemcu-build.com: nodemcu-master-6-modules-2016-12-18-20-22-54-integer.bin
    Required modules: bme280, (file), http, tmr, (uart), wifi
    --]]

    config = {
      ver = "fips_espdomocl_1, Dec 2016, www.4FipS.com",
      timer = { connect = 1, refresh = 2 },
      -- The below ones need to be customized: 'ssid, 'pwd', 'host' and 'sensor_idx'
      ap = { ssid = "Sesamstraat", pwd = "pwd" },
      domoticz = { host="10.0.0.213:8080", sensor_idx = 155, refresh_rate=60 },
      pin = { sda = 1, scl = 2 },
    }

    function connect(on_ok)
      print("CONNECT: "..config.ap.ssid)
      wifi.setmode(wifi.STATION)
      wifi.sta.config(config.ap.ssid, config.ap.pwd)
      wifi.sta.connect()
      tmr.alarm(config.timer.connect, 1000, 1, function()
        ip = wifi.sta.getip()
        if ip then
          tmr.stop(config.timer.connect)
          print("Connected: "..ip)
          if on_ok then on_ok() end
        else
          print("Connecting...")
        end
      end)
    end

    function loop()
      refresh()
      tmr.alarm(config.timer.refresh, config.domoticz.refresh_rate * 1000, 1, function()
        refresh()
      end)
    end

    function refresh()
      bme280.startreadout(0, function ()
        temp10x, _ = bme280.temp() / 10
        humi10x, _ = bme280.humi() / 100
        baro10x, _ = bme280.baro() / 100
        t, h, b = (temp10x/10).."."..(temp10x%10), (humi10x/10).."."..(humi10x%10), (baro10x/10).."."..(baro10x%10)
        print(string.format("BME280: T=%s C, H=%s%%, P=%s hPa", t, h, b))
        url = "http://%s/json.htm?type=command&param=udevice&idx=%d&nvalue=0&svalue=%s;%s;0;%s;0"
        req = string.format(url, config.domoticz.host, config.domoticz.sensor_idx, t, h, b)
        print("REQ: "..req)
        http.get(req, nil, function(code, data)
          if code < 0 then
            print("HTTP ERROR: "..code)
          else
            print("RES: "..code.." "..data)
          end
        end)
      end)
    end

    print(config.ver)
    bme280.init(config.pin.sda, config.pin.scl, nil, nil, nil, 0)
    connect(loop)

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Fri Mar 31, 2017 1:58 pm
by FipS
It seems there is a problem with getting data from the sensor. The Lua script panics (crashes) because the call to the function bme280.temp() returns nil, and nil / 10 results in the arithmetic error:

PANIC: unprotected error in call to Lua API (init.lua:45: attempt to perform arithmetic on a nil value)

I would recommend checking if the sensor is properly connected, and that the firmware contains the BME280 module (I believe the firmware prints out the list of included modules to the serial output each time ESP restarts).

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Mon Apr 03, 2017 9:19 pm
by FipS
Also, check the i2c address of your BME280 sensor. It seems that the NodeMCU firmware expects that the address is either 0x76 or 0x77 (BME280_I2C_ADDRESS). I can imagine there might be a BME280 sensor with a different address in which case bme280.temp() would return nil to indicate unsuccessful readout.

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Sat Apr 15, 2017 1:36 pm
by Pino
I found what I have done wrong. On my ESP you see only D0 D2 D2 D3 etc
And because I did not see D1 I used SD1 and SD2. But is seems to be a misprint. Now I have connected it to the D2 and D2 and it works. Thanks a lot for your help.

regards from Pino in The Netherlands.

Next question: is it possible to extend this project with a motion detection sensor like this one:
https://nl.aliexpress.com/item/1pcs-HC-SR501-Adjust-IR-Pyroelectric-Infrared-PIR-Motion-Sensor-Detector-Module-for-arduino-for-raspberry/32733268757.html?spm=2114.13010608.0.0.n5rWB6
And a smoke, CO2 sensors?

Re: Home automation with Domoticz, ESP8266 and BME280

PostPosted: Wed Apr 19, 2017 9:28 pm
by FipS
Good to hear it now works for you!

Regarding the PIR motion sensor, it should be pretty easy to hook it up to ESP8266. It uses just one pin named OUT to report its state. Basically, each time motion is detected the OUT pin goes HIGH. So it should be enough to just monitor changes on the pin, something like this:

Code: Select all
function motion()
  print("Motion detected!")
end

print("Ready...")
pin = 1
gpio.trig(pin, "up", motion)

With the sensor connected like this:

Code: Select all
NodeMCU      PIR
------------------
  D1 ....... OUT
  VCC ...... 3V3 (or VIN for higher voltage if necessary)
  GND ...... GND

The next step would be to pass this information to Domoticz. This could be done similarly as for BME280 by creating a virtual device, but this time the sensor type would be "Alert".

The url for triggering the Alert event would look something like this:
Code: Select all
http://10.0.1.20:8080/json.htm?type=command&param=udevice&idx=999&nvalue=4&svalue=Motion detected!

Other sensor types might be handled similarly, but be aware the more data the sensor provides the more complicated it is to hook it up, especially if there is no direct support for the sensor in the NodeMCU firmware.