ABB Aurora Web Inverter Monitor (WIM): Storage devices – 3

Renzo Mischianti

4 years ago

Spread the love

ABB PowerOne Aurora Web Inverter Centraline Storage devices

Another problem that we must solve is where storage all the data we need.

ABB Aurora inverter centraline software layer

As you can see from the schema I use two type of storage for some reasons that I’m going to explain better.

Toggle

SPIFFS

As described in deep in the article WeMos D1 mini (esp8266), integrated SPIFFS Filesystem esp8266 have internal flash that can be used to store data, but if you read the article you discover that have some limitations and It isn’t suitable to store logging data.

Here the code of the static content like HTML page and JS file:

bool handleFileRead(String path){  // send the right file to the client (if it exists)
  DEBUG_PRINT(F("handleFileRead: "));
  DEBUG_PRINTLN(path);

  if(path.endsWith("/")) path += F("index.html");           // If a folder is requested, send the index file
  String contentType = getContentType(path);             // Get the MIME type
  String pathWithGz = path + F(".gz");
  if(SPIFFS.exists(pathWithGz) || SPIFFS.exists(path)){  // If the file exists, either as a compressed archive, or normal
    if(SPIFFS.exists(pathWithGz))                          // If there's a compressed version available
      path += F(".gz");                                         // Use the compressed version
    fs::File file = SPIFFS.open(path, "r");                    // Open the file
    size_t sent = httpServer.streamFile(file, contentType);    // Send it to the client
    file.close();                                          // Close the file again
    DEBUG_PRINTLN(String(F("\tSent file: ")) + path + String(F(" of size ")) + sent);
    return true;
  }
  DEBUG_PRINTLN(String(F("\tFile Not Found: ")) + path);
  return false;                                          // If the file doesn't exist, return false
}

But this “filesystem” can be very usefully to put our WebServer pages that can only be readed, the only exception we add is the storing of IP address to expose.

Why FTP server to upload data to SPIFFS?

I add an FTP server to manage the data to SPIFFS because I’d like to change that data for testing new features or chart in the fastest way.

You can disable It with comment this define:

// Uncomment to enable server ftp.
#define SERVER_FTP

It’s use an external library very simple but for me very usefully, you check the library usage on my article “FTP server on esp8266 and esp32“.

The idea is that the web pages normally change few times, to SPIFFS It’s perfect, but remember that you haven’t folder, in my case I avoid this little problem with the creation of single package with JS (React/Redux) and some other stuff and so basically we have a webapp with simple structure.

In the futures article I’m going to explain a simple system to serve this webapp dinamically and manage gzip compression.

SD Card

Therefore we have solved the problem of where to put our WebApp, but the real “problem” is where store all logging data.

The solution is simple and consolidated, an SD card, exist various type of SD card and in particular there are some SD for Industrial use (usually they aren’t micro sd) tha grant durability, but a good microsd can be sufficient if you plain to do some backup every year.

Here the code to stream the data stored on SD to a response:

void streamFileOnRest(String filename){
	if (SD.exists(filename)){
		myFileSDCart = SD.open(filename);
		if (myFileSDCart){
			if (myFileSDCart.available()){
				DEBUG_PRINT(F("Stream file..."));
				DEBUG_PRINT(filename);
				httpRestServer.streamFile(myFileSDCart, F("application/json"));
				DEBUG_PRINTLN(F("...done."));
			}else{
				DEBUG_PRINTLN(F("Data not available!"));
				httpRestServer.send(204, F("text/html"), F("Data not available!"));
			}
			myFileSDCart.close();
		}else{
			DEBUG_PRINT(filename);
			DEBUG_PRINTLN(F(" not found!"));

			httpRestServer.send(204, F("text/html"), F("No content found!"));
		}
	}else{
		DEBUG_PRINT(filename);
		DEBUG_PRINTLN(F(" not found!"));
		httpRestServer.send(204, F("text/html"), F("File not exist!"));
	}
}

I write an article “How to use SD card with esp8266 and Arduino” to explain better how SD card work and how to connect It to the device.

Format of logging data

All data are stored in JSON format, this may seem like a wonderful solution when it comes to REST calls and data logging, but “not all that glitters is gold” (italian way of saying).

Pros

Data are structured, you can organize It in list “object” and other;
It’s simple to manage;
And data are ready for REST standard data format.

Cons

There is a very big problem: to generate a structured JSON the file must be on memory, and when we use an esp8266 the memory can be 40Kb available for the user (this mean 40.000/8 = 5000 characters) and when you do more than create a JSON file this size can be reduced a lot.

The alternative solution is to use a CSV format or other, but It’s complex to generate structured data and to update, and and with CSV you can add data without reload all the.

You can find more information and some example on my article “Manage JSON file with Arduino and esp8266“

Thanks

GitHub repository with all code BE and FE transpiled

Support forum

Spread the love