Problems after implementing weater functions


Laurent HOUSSAYE
 

Hello,

I have used OCS up to now with success with the Roof and Power functions only. I was able to activate 2 relays for the 12 V power supply of the telescope mount and accessories, and for the 220 V power supply (not wired yet at the relay output).

The roof commands are operating perfectly.

Recently I have included all the weather sensors allowed by the Weather code, using the examples you provided.

The code compiles and loads perfectly in the Arduino card after having included all the libraries required. I have put the file Charts.min.js in the SD card (renamed Charts.js) and inserted the SD card in the Ethernet shield as required.

But I have some problems : When I activate the 12 V power relay (ON), the Arduino reboots frequently (A
Watchdog ON) and that causes interruptions of this crucial power supply ! That's not acceptable as it is. When only the 220 V power supply relay is activated, no problem ! Very strange !

 

The weather sensors data are correctly displayed with the exception of the Sky Quality Meter that shows "Invalid" (However I have checked that TSL2591 responds at the correct address of 0x29 - checked with an I2C scan). See screen capture attached.

Last problem : the Weather charts are all empty. See pictures attached.

I would appreciate your help to be able to upgrade OCS with all its wonderful functions in a stable manner !

Best regards,

 

Laurent HOUSSAYE

 


Howard Dutton
 

On Tue, Apr 6, 2021 at 05:33 AM, Laurent HOUSSAYE wrote:

I have used OCS up to now with success with the Roof and Power functions only. I was able to activate 2 relays for the 12 V power supply of the telescope mount and accessories, and for the 220 V power supply (not wired yet at the relay output).

The roof commands are operating perfectly.

Recently I have included all the weather sensors allowed by the Weather code, using the examples you provided.

The code compiles and loads perfectly in the Arduino card after having included all the libraries required. I have put the file Charts.min.js in the SD card (renamed Charts.js) and inserted the SD card in the Ethernet shield as required.

But I have some problems : When I activate the 12 V power relay (ON), the Arduino reboots frequently (A Watchdog ON)

I'd turn the watchdog off, the feature is there but it working properly depends on Mega2560 firmware support that isn't always present.

and that causes interruptions of this crucial power supply ! That's not acceptable as it is. When only the 220 V power supply relay is activated, no problem ! Very strange !


Sounds like a hardware problem.  There are a wide range of effects with power supply sag, back EMF from the relay coils, RFI noise from the relay contacts, HV traveling on the surface of PCB's.

The weather sensors data are correctly displayed with the exception of the Sky Quality Meter that shows "Invalid" (However I have checked that TSL2591 responds at the correct address of 0x29 - checked with an I2C scan). See screen capture attached.

I provided an example that works for myself and others.  If you can get the SQM function working properly with other software do so then integrate that software.


Last problem : the Weather charts are all empty. See pictures attached.

Not sure why that'd be, obviously it works for others.  Perhaps give it time to write the next days log file, all these crashes could be causing an issue.

Sorry but implementing an OCS with all features is a complex thing with many ways for things to go wrong.


Laurent HOUSSAYE
 

Good morning Howard
the problem is that if I turn off the Watchdog the OCS web page fails and freezes...
I had no relay problem at all with my previous configuration and the same relay wiring (8xrelays card is powered separately and not by the MEGA 5V).
I have used the SQM code that you provided in the Weather Functions file for the TSL2591, exactly copied. Remark: It is the clone version of the Adafruit chip. 

I will attempt to use an original Adafruit chip, after first disconnecting this sensor and see if the problem is still present.
it would already be nice if it worked with all the other sensors !!!
Clear skies !
Laurent


Laurent HOUSSAYE
 

Some update after further testing :
I tried to put SQM OFF in config.h : no success : frequent reboots whatever the status of the 2 power relays (On or Off). No visible sign of these reboots when the relays are OFF (they remain OFF) except that the Up Time stays at 00000 min.
I tried to put all the I2C bus sensors OFF in config.h :
  • with the SDA and SCL wires still connected, same instabilities (in less than 1 minute)
  • With the SDA and SCL wires unplugged : stable ! No reboot at all, the 12V power relay stays ON at least up to 10 minutes "Up Time" ! Rain sensor and wind speed are correctly displayed however the weather log charts remain empty...
I need to try disconnecting only the SQM sensor to see if it is the culprit on the I2C bus !
What remains a bit strange to me yet is the fact that when I returned to my initial set up with no weather function activated (only ROR panel and Power relays panel), the system was stable even with the I2C SDA and SCL wires plugged in...
Is there a mix of software and hardware causes ?
I will let you aware of the future outcomes.
Regards,
Laurent


Laurent HOUSSAYE
 

Update of 18th of April 2021:
Starting point : OCS version 2.31.
  • Basic configuration : Roof control function + Power relays control function. Operates perfectly in particular the relays operate perfectly. That would eliminate the suspicion of a hardware problem (8 relays module powered indepedently of the MEGA).
  • Upgraded configuration (Basic plus Weather function - all sensors excepted SQM) : Weather data are correctly displayed on the overview panel but the Weather and Sky plots are displayed completely empty : the frames are displayed but nothing inside. See pictures in my previous post. The system turns unstable in this configuration.
- Unstable (no more than 1 minute Up Time) if I activate the power relays.
- Even if I do not activate the power relays, the system hangs and reboots, but it takes more time (Up Time reset during my lunch time, then reset in less than 10 minutes now that I am attending).
I tried to put WATCHDOG OFF as you suggested and it is worse : The OCS web page freezes and can no longer be opened. I returned to WATCHDOG ON_CC. I suppose that Up Time counter lets me know the time since the last hang-up and reboot.
I have copied exactly the arduino codes for the weather sensors that you have profided in the file named "Functions" under "Weather.ino examples".
Is there a way of debugging all this ?
Next thing I will do is connect an original TSL2591 sensor from Adafruit and try to get correct SQM data but I am getting short of ideas to solve this issue of instability.
Any help would be appreciated.
Best Regards,
Laurent


Laurent HOUSSAYE
 

Sorry I was wrong in my last post : the Basic configuration that is OK and perfectly stable is with OCS version 2.1c. I did'nt update it because it is operating perfectly.


Laurent HOUSSAYE
 

Last update for today @ 16:30 h UT:
Good point for me : I think I have found the cause of the instabilities and cured them.
Bad point for you Howard : It seems to be the latest version of OCS (2.31).
Let me explain : I implemented the Weather functions and connected all the sensors except the SQM sensor TSL2591 in my currently operating OCS version 2.1c, and not in the latest version that I had downloaded when I first attempted to implement the Weather functions. And now it works stable ! No hang up or reboot during at least 144 minutes (85 minutes with the relaus OFF and 59 minutes with the relay ON, as you can see !



The problem of frequent hang up and reboot seems had disapeared and seems then to be caused by using the latest version of OCS (3.21).
The Weather plots are still empty. Here I have made no progress... I will see tomorrow after more time spent. What is the supposed frequency of the updates for these plots ?
I will keep you also informed when I try to connect the original Adafruit TSL2591 in place of the clone one, in order to get this SQM function operate.
Best regards,
Laurent


Laurent HOUSSAYE
 

Sorry Howard, I should not have gone so quick, I have to amend the picture... Soon after having sent this last message, I realized that the Web page displayed on my screen was in fact frozen... No update possible, no web communication. I had to manually push the reset button on the Internet shield of the Arduino card.
Then the summary of my investigation up to now : Version 2.1c is much more stable (no spurious change in the power relay outputs, no hang up for at least 144 minutes) but 2 problems still present :
  • weather plots are empty
  • the web page freezes and is not reachable, which mandates a manual push of the internet shield reset button (and a power relay cycling ON / OFF / ON... Should I try with the WATCHDOG ON ? There is no option tu choose ON_CC in this version 2.1c of OCS. Only good point : the freezing of the web page and web communication (input & output) has no consequence on the power relay state (no spurious change). Only pushing the reset button on the internet shield causes a cycling of the power relay and a temporary loss of this critical power supply...
Please confirm me what version of OCS you have, running in your observatory set-up, and if it is possible for you to send me an exact copy for me to test it ?
Best regards,
Laurent


Howard Dutton
 

I just uploaded/tested the latest version (github) to take care of Koal01's issue, no surprise it came right up without issue.  Note that 2.31 and 2.32 are for our purposes identical and I did testing on both.

In addition to the advice I posted before... I suggest you simplify, remove/comment out functionality (in Weather.ino) until you reach stability then add features back one at a time until you find what is breaking it.

Charts not coming up can be due to not using the correct charts.min.js as described in the Config.h file.  Also, if something about the SD is broken (not reading the file, not writing the logs.)  Or even if something was botched while the browser got that Charts.js file for the first time and you have a corrupt copy in the browser cache (i.e. clear the browser cache.)

For switching critical power relays such as what you describe I setup so it defaults normally closed when no control from the OCS is present, so it can reboot without those troubles.  Even the cheapo ebay "8 Relay Module" sitting on my desk for testing allows that for any relay.





I also enabled the other sensors you have, no code added to Weather.ino for them though as I don't have them wired up here:


Laurent HOUSSAYE
 

Hello everybody
Thanks Howard for your efforts: I downloaded the latest OCS version 2.33, and set it up for my observatory
  1. with my basic configuration in config.h with no Weather function : OK
  2. as 1. plus outside Temperature : OK
  3. as 2. plus Barometric pressure : OK
  4. as 3. plus humidity : OK for 14 minutes at least
  5. as 4. plus windspeed : OK for 17 minutes at least
  6. as 5. plus rain sensor : Web page freezes (communication lost) after 9 minutes. Automatic reboot (Watchdog set @ ON_CC), then stable for 20 minutes, then 1 minute : very variable. Not sure that the rain sensor is the cause because I had this type of freeze with config 3 quite rapidly and after reboot it remained stable for some 10 minutes, and I went on further with next configurations...

You are right for the relays. I returned to HIGH in lines 110 to 124 of OCS.ino so that it defaults normally closed when no control from the OCS is present, so it effectively reboots without troubles. However it reboots, quite randomly.
     7. as 6. plus Weather charts ON : No change in the unsatbility status and charts are all empty. I will try to reload the Chart.js file but I am pretty sure i had the correct one in the SD card root directory. It does not need to be executed or run ? Just put in the SD card and that's all ?
    8. As 6. plus sky IR temperature : still random reboots (perhaps a bit more frequent - my feeling) but all sensors that are physically connected to OCS are displayed correctly :


Are there any debugging codes that could be useful for me to improve my investigation ?

That's all for now.
Best regards,
Laurent


Howard Dutton
 

On Tue, Apr 20, 2021 at 11:08 AM, Laurent HOUSSAYE wrote:
Are there any debugging codes that could be useful for me to improve my investigation ?
Not really, it seems to be a sensors problem, perhaps down in the libraries.

What libraries has not been discussed.


Howard Dutton
 
Edited

On Tue, Apr 20, 2021 at 11:08 AM, Laurent HOUSSAYE wrote:
     7. as 6. plus Weather charts ON : No change in the unsatbility status and charts are all empty. I will try to reload the Chart.js file but I am pretty sure i had the correct one in the SD card root directory. It does not need to be executed or run ? Just put in the SD card and that's a
The OCS reads the "Chart.js" file and sends it to your web browser to run.

I commented on the likely causes already.  I just started from scratch here in that last go around, my development laptop changed so all new libraries/downloads.  Worked fine here should work for you... same hardware + same software = same result.  If the result isn't the same, something is different.


Howard Dutton
 

On Tue, Apr 20, 2021 at 11:35 AM, Howard Dutton wrote:
Not really, it seems to be a sensors problem, perhaps down in the libraries.
Or perhaps corrupt signaling on the I2C bus locks up the (Arduino) Wire library.


Howard Dutton
 

I would tackle problems one at a time...

Get the rain sensor working first, that should be a basic device right?


Howard Dutton
 

What about that I2C bus...

Bus and all sensors are 5V rated/compatible?  Level translation for 3V3?
Loading resistors?
Twisted pair? How long?


Laurent HOUSSAYE
 

When the relays are HIGH (220 V AC in Open Circuit at the relay ouput terminals) the system is stable : no reboot in more than 88 minutes). See screen copy :


I let it in this state for the night and will see tomorrow morning.


Howard Dutton
 
Edited

On Tue, Apr 20, 2021 at 12:51 PM, Laurent HOUSSAYE wrote:
When the relays are HIGH (220 V AC in Open Circuit at the relay ouput terminals) the system is stable : no reboot in more than 88 minutes). See screen copy
If you're saying switching mains (or even the act of switching relays) is crashing the system go back to my first post in reply to this.

Interference:
EMI
RF
Back EMF.(flyback from relay coils.)
High voltage traveling condensation/surfaces.
(etc.)


Laurent HOUSSAYE
 

Good morning,
Status of this morning : There are still reboots, even with the the relays HIGH (i.e. no  220 V AC current transiting the output terminals of the relay).
Let me make it clearer : switching the relays has never caused the system to crash. I had just the impression that the system stability could be affected by the fact that current goes through the relay output terminals or not. If I had a hardware problem with my 8xrelays card, how would my initial set-up (with WEATHER OFF) be completely stable whatever the status of the relays ?
To go slowly I switched OFF all the sensors of the I2c bus in config.h. They are still physically connected. I just kept ON the rain and winspeed sensors.

I have also reformatted the SD card (Fat32 - no quick formating) and placed a freshly downloaded Chart.min.js from the page indicated in config.h, renamed Chart.js, in the root directory.
I am going to put WEATHER CHARTS ON to see if something is displayed. Done, System sttill stable but charts are still empty :


Despite these empty charts, the system is stable since 153 minutes, with the 220 V AC current passing through the relay output terminals :

I need to get these charts correct with these very basic sensors connected before I go further, isn't it ? What would you suggest as further action ?
Best regards,
Laurent


Howard Dutton
 

A statement like this:

"When the relays are HIGH (220 V AC in Open Circuit at the relay ouput terminals) the system is stable : no reboot in more than 88 minutes). See screen copy"
Leads me to think you were implying the relay state could impact stability.


Howard Dutton
 

On Wed, Apr 21, 2021 at 06:50 AM, Laurent HOUSSAYE wrote:
I need to get these charts correct with these very basic sensors connected before I go further, isn't it ? What would you suggest as further action ?
Check the SD card contents to see that the log files are being written.  They are plain text files, you can open them in notepad on a PC.

Read up on the Arduino SD card support, what works and what doesn't.  Might be the kind of SD card.  How it is formatted (some say windows format doesn't work I've found it is fine.)  Etc.

Attempt to run another example Sketch to see the SD card work using the same libraries the OCS uses.