esp32 / ESP32 Tutorial / esp8266 / esp8266 tutorial / Flash memories storages / How to use external SPI Flash memories / SPI
5

esp32 and esp8266: FAT filesystem on external SPI flash memory

by Renzo Mischianti · Published 17 January 2023 · Updated 11 February 2024

Spread the love

We had already explained internal flash with various filesystems and SD management. The first integrated with low energy consumption, the second from small to huge capacity but quite thirsty for energy.

But why now I’d like to explain external SPI flash if these devices already have integrated one?
The answer is pretty simple: internal flash has a short life for the write cycle, but flash memories like Winbond (or other) exist that, instead of having one hundred thousand cycles, have millions of write cycles and can be used like data logging.

So today, we are going to see the SPI Flash memory (NOR Flash). They are a single chip that can be managed via SPI and have high-speed access and low power consumption.

Flash memory is an electronic non-volatile computer memory storage medium that can be electrically erased and reprogrammed. The two main types of flash memory, NOR flash and NAND flash, are named for the NOR and NAND logic gates. NAND flash and NOR flash use the same cell design, consisting of floating gate MOSFETs. They differ at the circuit level: in NAND flash, the relationship between the bit line and the word lines resembles a NAND gate; in NOR flash, it resembles a NOR gate; this depends on whether the state of the bit line or word lines is pulled high or low.

Flash memory, a type of floating-gate memory, was invented at Toshiba in 1980 and is based on EEPROM technology. Toshiba began marketing flash memory in 1987. EPROMs had to be erased completely before they could be rewritten. NAND flash memory, however, may be erased, written, and read in blocks (or pages), which generally are much smaller than the entire device. NOR flash memory allows a single machine word to be written – to an erased location – or read independently. A flash memory device typically consists of one or more flash memory chips (each holding many flash memory cells), along with a separate flash memory controller chip.
Wikipedia

SPI Flash memories pinout

There are SMD, and Discrete IC managed by SPI protocol.

Here a set of SPI flash with different size w25q16 SMD 2Mb - w25q16 Discrete 2Mb - w25q32 SMD 4Mb - w25q32 Discrete 4Mb - w25q64 SMD 8Mb - w25q64 Discrete 8Mb - w25q128 SMD 16Mb - w25q128 Discrete 16Mb W25Q32 W25Q64 w25q128 module 4Mb 8Mb 16Mb

esp8266 wiring diagram

As usual, for the esp8266 device, I use WeMos D1 mini.

WeMos D1 mini esp8266 pinout mischianti low resolution

Arduino	SPI Flash
D8	/CS	Pulled UP if not standard CS
D6	DI (IO₁)
D7	DI (IO₀)
D5	CLK
3.3v	/WP
3.3v	/Hold
GND	GND
3.3v	VCC

Here is the result on the breadboard.

esp8266 WeMos D1 mini connection DIP8 SPI Flash breadboard

And here is the schema.

esp8266 WeMos D1 mini connection DIP8 SPIFlash schema

esp32 connection schema

Here the connection schema

esp32 DOIT DEV KIT v1 connection DIP8 SPIFlash breadboard w25q128

Arduino	SPI Flash
GPIO5	/CS	Pulled UP if not standard CS
GPIO19	DI (IO₁)
GPIO23	DI (IO₀)
GPIO18	CLK
3.3v	/WP
3.3v	/Hold
GND	GND
3.3v	VCC

Here is the schema.

esp32 DOIT DEV KIT v1 connection DIP8 SPI Flash schema w25q64

SPI Flash basic usage

If you don’t need a particular data structure, you can do basic and raw usage of the SPI Flash.

For primary usage, I advise using a library named SPIMemory, quite simple but with good support without complexity. You can download It via library manager from Arduino IDE.

For additional information on this library, you can see the article “Arduino: external fast SPI Flash memory“.

SPIMemory library from Arduino IDE library manager

Pay attention to esp32. You must set SPI frequencies limits to 24Mhz; esp8266 had no limitations.

To set frequencies limits, use a command like this (read the Arduino article to get more information):

flash.setClock(24000000);

Here is a simple example that stores a JSON string in the initial address 0, and I re-read It, then I ask the method getAddress the first available contiguous position where I can store another JSON string, and I save It re-read.

/*
 *  Manage external SPI Flash with Arduino
 *  Write and read a JSON structure like a String,
 *  find first address available
 *  and write and read another JSON structure
 *
 *  with library SPIMemory
 *
 *  by Mischianti Renzo <https://mischianti.org>
 *
 *  https://mischianti.org/
 *
 *	SPIFlash connected via SPI standard
 *
 */

#include <SPIMemory.h>
#include <ArduinoJson.h>

//SPIFlash flash; // If you don't specify the library use standard SPI connection
SPIFlash flash(SS, &SPI);
void setup() {
  Serial.begin(115200);

  while (!Serial) ; // Wait for Serial monitor to open
  delay(100);

  // flash.setClock(12000000); // uncomment here for Arduino SAMD boards

  flash.begin(); // If SPIMemory isn't recognized you can specify the size of memory

  // flash.eraseChip();

  Serial.print(F("Flash size: "));
  Serial.print((long)(flash.getCapacity()/1000));
  Serial.println(F("Kb"));

  unsigned long strAddr = 0;
  unsigned long strAddrSecondString = 0;

  Serial.println();

  // Allocate a temporary JsonDocument
  // Don't forget to change the capacity to match your requirements.
  // Use arduinojson.org/v6/assistant to compute the capacity.
  //  StaticJsonDocument<512> doc;
  // You can use DynamicJsonDocument as well
  Serial.println(F("Generate JSON file!"));
  DynamicJsonDocument doc(512);

  // Set the values in the document
  doc["energyLifetime"] = 21698620;
  doc["energyYearly"] = 1363005;

  Serial.print(F("Put data in a buffer.. "));
  // Serialize JSON to file
  String buf;
  if (serializeJson(doc, buf) == 0) {
	  Serial.println(F("failed to write buffer"));
  }

  if (flash.writeStr(strAddr, buf)){
	  Serial.print(F("OK, writed on address "));
	  Serial.println(strAddr);
  }else{
	  Serial.println(F("KO"));
  }

  String outputString = "";
  if (flash.readStr(strAddr, outputString)) {
    Serial.print(F("Read json: "));
    Serial.println(outputString);
    Serial.print(F("From address: "));
    Serial.println(strAddr);
  }


  Serial.println(F("Generate JSON file!"));
  DynamicJsonDocument doc2(512);

  // Set the values in the document
  doc2["energyLifetime"] = 222;
  doc2["energyYearly"] = 333;


  Serial.println();
  Serial.print(F("Check first free sector: "));
  strAddrSecondString = flash.getAddress(doc2.size());
  Serial.println(strAddrSecondString);
  Serial.println();

  Serial.print(F("Stream data in flash memory!"));

  Serial.print(F("Put data in a buffer.."));
  // Serialize JSON to file
  String buf2;
  if (serializeJson(doc2, buf2) == 0) {
	  Serial.println(F("failed to write buffer"));
  }
    // Print test file

  if (flash.writeStr(strAddrSecondString, buf2)){
	  Serial.print(F("OK, writed on address "));
	  Serial.println(strAddrSecondString);
  }else{
	  Serial.println(F("KO"));
  }

  String outputString2 = "";
  if (flash.readStr(strAddrSecondString, outputString2)) {
    Serial.print(F("Read data: "));
    Serial.println(outputString2);
    Serial.print(F("From address: "));
    Serial.println(strAddrSecondString);
  }

  while (!flash.eraseSector(strAddr));
  while (!flash.eraseSector(strAddrSecondString));

}

void loop() {

}

Here is the console result.

Flash size: 8388Kb

Generate JSON file!
Put data in a buffer.. OK, writed on address 0
Read json: {"energyLifetime":21698620,"energyYearly":1363005}
From address: 0
Generate JSON file!

Check first free sector: 56

Stream data in flash memory!Put data in a buffer..OK, writed on address 56
Read data: {"energyLifetime":222,"energyYearly":333}
From address: 56

SPI Flash advanced use with SdFat filesystem

A more complex library (and more hungry for resources) The Adafruit one, the Adafruit SPIFlash, needs to be used with the SdFat Adafruit fork.

This library Is discouraged on Arduino’s boards like UNO or Mega because only for buffer, It needs 512 bytes, and every file has some byte of a preamble, and you risk wasting a lot of RAM.
For a low-power microcontroller, It’s better to use a library like SPIMemory.

You can also find these libraries on the library manager “Adafruit SPIFlash” and SdFat – Adafruit Fork”.

Arduino IDE library manager Adafruit SPIFlash and SdFat fork

Add new SPI Flash type

The set of chips is supported. It’s limited to a specified list but create a new device. It’s pretty simple. For example, I have a W25X80AVAIZ (I think Winbond clone), a discrete component buy for a very low price.
It has these characteristics:

1Mb capacity;
104Mhz of speed;
Winbond clone;
Single dual and quad SPI.

Then, I write a simple sketch to check the compatibility:

/*
 *  Retrieve basic core info of Flash SPI
 *  Add a custom device w25x80
 *  library Adafruit_SPIFlash and SdFat - AdafruitFork
 *
 *  by Mischianti Renzo <https://mischianti.org>
 *
 *  https://mischianti.org/
 *
 *	SPIFlash connected via SPI standard check wiring on the article
 *
 */

#include "SdFat.h"
#include "Adafruit_SPIFlash.h"

Adafruit_FlashTransport_SPI flashTransport(SS, SPI); // Set CS and SPI interface
Adafruit_SPIFlash flash(&flashTransport);

void setup()
{
  Serial.begin(115200);
  while ( !Serial ) delay(100);   // wait for native usb

  if (flash.begin()) {
	  Serial.println(F("Device finded and supported!"));
  } else {
	  Serial.println(F("Problem to discover and configure device, check wiring also!"));
  }
  // Set 4Mhz SPI speed
  flashTransport.setClockSpeed(4000000, 4000000); // added to prevent speed problem

  Serial.println();

  Serial.println("Adafruit Serial Flash get basic info: ");
  Serial.print("JEDEC ID (FFFFFF for unknown): "); Serial.println(flash.getJEDECID(), HEX);
  Serial.print("Flash size: "); Serial.println(flash.size());

  Serial.println();Serial.println();

  uint8_t jedec_ids[4];
  flashTransport.readCommand(SFLASH_CMD_READ_JEDEC_ID, jedec_ids, 4);

  // For simplicity with commonly used device, we only check for continuation
  // code at 2nd byte (e.g Fujitsu FRAM devices)
  if (jedec_ids[1] == 0x7F) {
    // Shift and skip continuation code in 2nd byte
    jedec_ids[1] = jedec_ids[2];
    jedec_ids[2] = jedec_ids[3];
  }

  Serial.println("Retrieve JDEC_ID");

  Serial.print("Manufacturer ID: 0x");
  Serial.println(jedec_ids[0], HEX);

  Serial.print("Memory Type: 0x");
  Serial.println(jedec_ids[1], HEX);

  Serial.print("Capacity: 0x");
  Serial.println(jedec_ids[2], HEX);
  Serial.print("Capacity DEC: ");
  Serial.println(jedec_ids[2], DEC);
}

void loop()
{
  // nothing to do
}

The serial output for my unsupported device is:

Unknown flash device 0xEF4014
Problem to discover and configure device, check wiring also!

Adafruit Serial Flash get basic info: 
JEDEC ID (FFFFFF for unknown): FFFFFF
Flash size: 0 


Retrieve JDEC_ID
Manufacturer ID: 0xEF
Memory Type: 0x40 
Capacity: 0x14
Capacity DEC: 20

So I get from the Adafruit_SPIFlash library the file flash_devices.h, and I get the most similar device is in the list.

Then I use:

Capacity DEC to get the size: .total_size = (1UL << 20), /* 1 MiB */
Manufacturer ID for the manufacturer: .manufacturer_id = 0xef
Memory type for the memory_type: .memory_type = 0x40
The capacity for capacity: .capacity = 0x14

I try to find some information on the compatible device datasheet.

  // Settings for the Winbond W25Q80BV (1 MiB SPI flash.)
  // https://pdf1.alldatasheet.com/datasheet-pdf/view/555565/WINBOND/W25Q80BV.html
  #define W25X80BV                                                            \
    {                                                                            \
      .total_size = (1UL << 20), /* 1 MiB */                                     \
          .start_up_time_us = 5000, .manufacturer_id = 0xef,                     \
      .memory_type = 0x40, .capacity = 0x14, .max_clock_speed_mhz = 104,         \
      .quad_enable_bit_mask = 0x02, .has_sector_protection = false,              \
      .supports_fast_read = true, .supports_qspi = true,                         \
      .supports_qspi_writes = true, .write_status_register_split = false,        \
      .single_status_byte = false, .is_fram = false,                             \
    }


  static const SPIFlash_Device_t possible_devices[] = {
		  W25X80BV,
  };

  // Initialize flash library and check its chip ID.
  if (!flash.begin(possible_devices)) {
    Serial.println("Error, failed to initialize flash chip!");
    while(1) yield();
  }

Now the complete test sketch become:

/*
 *  Retrieve basic core info of Flash SPI
 *  Add a custom device w25x80
 *  library Adafruit_SPIFlash and SdFat - AdafruitFork
 *
 *  by Mischianti Renzo <https://mischianti.org>
 *
 *  https://mischianti.org/
 *
 *	SPIFlash connected via SPI standard check wiring on the article
 *
 */

#include "SdFat.h"
#include "Adafruit_SPIFlash.h"

Adafruit_FlashTransport_SPI flashTransport(SS, SPI); // Set CS and SPI interface
Adafruit_SPIFlash flash(&flashTransport);

void setup()
{
  Serial.begin(115200);
  while ( !Serial ) delay(100);   // wait for native usb

  // Settings for the Winbond W25Q80BV (1 MiB SPI flash.)
  // https://pdf1.alldatasheet.com/datasheet-pdf/view/555565/WINBOND/W25Q80BV.html
  #define W25X80BV                                                            \
    {                                                                            \
      .total_size = (1UL << 20), /* 1 MiB */                                     \
          .start_up_time_us = 5000, .manufacturer_id = 0xef,                     \
      .memory_type = 0x40, .capacity = 0x14, .max_clock_speed_mhz = 104,         \
      .quad_enable_bit_mask = 0x02, .has_sector_protection = false,              \
      .supports_fast_read = true, .supports_qspi = true,                         \
      .supports_qspi_writes = true, .write_status_register_split = false,        \
      .single_status_byte = false, .is_fram = false,                             \
    }


  static const SPIFlash_Device_t possible_devices[] = {
		  W25X80BV,
  };

  if (flash.begin(possible_devices)) {
	  Serial.println(F("Device finded and supported!"));
  } else {
	  Serial.println(F("Problem to discover and configure device, check wiring also!"));
  }
  // Set 4Mhz SPI speed
  flashTransport.setClockSpeed(4000000, 4000000); // added to prevent speed problem

  Serial.println();

  Serial.println("Adafruit Serial Flash get basic info: ");
  Serial.print("JEDEC ID (FFFFFF for unknown): "); Serial.println(flash.getJEDECID(), HEX);
  Serial.print("Flash size: "); Serial.println(flash.size());

  Serial.println();Serial.println();

  uint8_t jedec_ids[4];
  flashTransport.readCommand(SFLASH_CMD_READ_JEDEC_ID, jedec_ids, 4);

  // For simplicity with commonly used device, we only check for continuation
  // code at 2nd byte (e.g Fujitsu FRAM devices)
  if (jedec_ids[1] == 0x7F) {
    // Shift and skip continuation code in 2nd byte
    jedec_ids[1] = jedec_ids[2];
    jedec_ids[2] = jedec_ids[3];
  }

  Serial.println("Retrieve JDEC_ID");

  Serial.print("Manufacturer ID: 0x");
  Serial.println(jedec_ids[0], HEX);

  Serial.print("Memory Type: 0x");
  Serial.println(jedec_ids[1], HEX);

  Serial.print("Capacity: 0x");
  Serial.println(jedec_ids[2], HEX);
  Serial.print("Capacity DEC: ");
  Serial.println(jedec_ids[2], DEC);


}

void loop()
{
  // nothing to do
}

And now the result is:

Device finded and supported!

Adafruit Serial Flash get basic info: 
JEDEC ID (FFFFFF for unknown): EF4014
Flash size: 1048576


Retrieve JDEC_ID
Manufacturer ID: 0xEF
Memory Type: 0x40
Capacity: 0x14
Capacity DEC: 20

Test and Troubleshooting

To check if all It’s ok, you can use the test given in the library. First, you must launch the example SdFat_speedtest, for Arduino UNO like board with little RAM, you must reduce the buffer from

#define BUFSIZE   4096

#define BUFSIZE   512

here is the complete sketch with custom IC:

// The MIT License (MIT)
// Copyright (c) 2019 Ha Thach for Adafruit Industries

#include "SdFat.h"
#include "Adafruit_SPIFlash.h"

// Uncomment to run example with custom SPI and SS e.g with FRAM breakout
 #define CUSTOM_CS   SS
 #define CUSTOM_SPI  SPI
#define EXTERNAL_FLASH_USE_QSPI

  Adafruit_FlashTransport_SPI flashTransport(CUSTOM_CS, CUSTOM_SPI);

  Adafruit_SPIFlash flash(&flashTransport);

#define BUFSIZE   4096

// 4 byte aligned buffer has best result with nRF QSPI
uint8_t bufwrite[BUFSIZE] __attribute__ ((aligned(4)));
uint8_t bufread[BUFSIZE] __attribute__ ((aligned(4)));

// the setup function runs once when you press reset or power the board
void setup()
{
  Serial.begin(115200);
  while ( !Serial ) delay(100);   // wait for native usb
  SPI.setFrequency(12000000);

  // Settings for the Winbond W25Q80BV (1 MiB SPI flash.)
  // https://pdf1.alldatasheet.com/datasheet-pdf/view/555565/WINBOND/W25Q80BV.html
  #define W25X80BV                                                            \
    {                                                                            \
      .total_size = (1UL << 20), /* 1 MiB */                                     \
          .start_up_time_us = 5000, .manufacturer_id = 0xef,                     \
      .memory_type = 0x40, .capacity = 0x14, .max_clock_speed_mhz = 104,         \
      .quad_enable_bit_mask = 0x02, .has_sector_protection = false,              \
      .supports_fast_read = true, .supports_qspi = true,                         \
      .supports_qspi_writes = true, .write_status_register_split = false,        \
      .single_status_byte = false, .is_fram = false,                             \
    }


  /// List of all possible flash devices used by Adafruit boards
  static const SPIFlash_Device_t possible_devices[] = {
      // Main devices used in current Adafruit products
		  W25X80BV,
  };

//  flash.begin(possible_devices); // custom board
  flash.begin(); // start with standard board

  pinMode(LED_BUILTIN, OUTPUT);
  flash.setIndicator(LED_BUILTIN, true);

  Serial.println("Adafruit Serial Flash Speed Test example");
  Serial.print("JEDEC ID: "); Serial.println(flash.getJEDECID(), HEX);
  Serial.print("Flash size: "); Serial.println(flash.size());
  Serial.flush();

  write_and_compare(0xAA);
  write_and_compare(0x55);

  Serial.println("Speed test is completed.");
  Serial.flush();
}

void print_speed(const char* text, uint32_t count, uint32_t ms)
{
  Serial.print(text);
  Serial.print(count);
  Serial.print(" bytes in ");
  Serial.print(ms / 1000.0F, 2);
  Serial.println(" seconds.");

  Serial.print("Speed : ");
  Serial.print( (count / 1000.0F) / (ms / 1000.0F), 2);
  Serial.println(" KB/s.\r\n");
}

bool write_and_compare(uint8_t pattern)
{
  uint32_t ms;

  Serial.println("Erase chip");
  Serial.flush();

#define TEST_WHOLE_CHIP

#ifdef TEST_WHOLE_CHIP
  uint32_t const flash_sz = flash.size();
  flash.eraseChip();
#else
  uint32_t const flash_sz = 4096;
  flash.eraseSector(0);
#endif

  flash.waitUntilReady();

  // write all
  memset(bufwrite, (int) pattern, sizeof(bufwrite));
  Serial.printf("Write flash with 0x%02X\n", pattern);
  Serial.flush();
  ms = millis();

  for(uint32_t addr = 0; addr < flash_sz; addr += sizeof(bufwrite))
  {
    flash.writeBuffer(addr, bufwrite, sizeof(bufwrite));
  }

  uint32_t ms_write = millis() - ms;
  print_speed("Write ", flash_sz, ms_write);
  Serial.flush();

  // read and compare
  Serial.println("Read flash and compare");
  Serial.flush();
  uint32_t ms_read = 0;
  for(uint32_t addr = 0; addr < flash_sz; addr += sizeof(bufread))
  {
    memset(bufread, 0, sizeof(bufread));

    ms = millis();
    flash.readBuffer(addr, bufread, sizeof(bufread));
    ms_read += millis() - ms;

    if ( memcmp(bufwrite, bufread, BUFSIZE) )
    {
      Serial.printf("Error: flash contents mismatched at address 0x%08X!!!", addr);
      for(uint32_t i=0; i<sizeof(bufread); i++)
      {
        if ( i != 0 ) Serial.print(' ');
        if ( (i%16 == 0) )
        {
          Serial.println();
          Serial.printf("%03X: ", i);
        }

        Serial.printf("%02X", bufread[i]);
      }

      Serial.println();
      return false;
    }
  }

  print_speed("Read  ", flash_sz, ms_read);
  Serial.flush();

  return true;
}

void loop()
{
  // nothing to do
}

If you have a Serial output like this (I use […] to indicate the line replication):

Adafruit Serial Flash Speed Test example
JEDEC ID: EF4014
Flash size: 1048576
Erase chip
Write flash with 0xAA
Write 1048576 bytes in 1.03 seconds.
Speed : 1016.06 KB/s.

Read flash and compare
Error: flash contents mismatched at address 0x00000000!!!
000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
[...]
FF0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Erase chip
Write flash with 0x55
Write 1048576 bytes in 1.03 seconds.
Speed : 1017.05 KB/s.

Read flash and compare
Error: flash contents mismatched at address 0x00000000!!!
000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
[...]
FF0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Speed test is completed.

The lines (or other HEX characters) with

000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

indicate that the data compared don’t match, so (If the previous test sketch work) probably you had some problem.

In most cases, the SPI speed is too high; this does not mean that the speed is too high for the microcontroller or IC, but it can be a problem with a poor connection cable.

But the solution to this was to reduce the speed, so you must go to add a line like this after the flash.begin():

flashTransport.setClockSpeed(24000000, 24000000);

esp32 WeMos LOLIN32 external SPI Flash memory on breadboard — esp32 WeMos LOLIN32 external SPI Flash memory on the breadboard

the command set the SPI speed to 24Mhz, and now the result become like so:

Adafruit Serial Flash Speed Test example
JEDEC ID: EF4014
Flash size: 1048576
Erase chip
Write flash with 0xAA
Write 1048576 bytes in 4.07 seconds.
Speed : 257.38 KB/s.

Read flash and compare
Read  1048576 bytes in 0.47 seconds.
Speed : 2226.28 KB/s.

Erase chip
Write flash with 0x55
Write 1048576 bytes in 4.07 seconds.
Speed : 257.38 KB/s.

Read flash and compare
Read  1048576 bytes in 0.47 seconds.
Speed : 2231.01 KB/s.

Speed test is completed.

If no compare errors are shown, the SPI Flash was correctly configured.

esp32 limitations

For esp32, the maximum stable speed I have reached is 24Mhz, so I must add

flashTransport.setClockSpeed(24000000, 24000000);

esp8266 limitations

esp8266 has a lower CPU frequency but no limitations on SPI frequencies.

WeMos D1 mini external SPI Flash memory on breadboard — WeMos D1 mini external SPI Flash memory on the breadboard

Practical sketch

First of all, you must Format the SPI Flash; in the examples of Adafruit_SPIFlash, you can find the SdFat_format example, that format the SPI flash for you.

When you launch the sketch, It asks you to insert, in the Serial monitor, “OK” in camel case, here the Serial output result.

Adafruit SPI Flash FatFs Format Example
Flash size: 1024 KB
Flash chip JEDEC ID: 0xEF4014
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
This sketch will ERASE ALL DATA on the flash chip and format it with a new filesystem!
Type OK (all caps) and press enter to continue.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

>>Send to COM17: "OK"<<
Creating and formatting FAT filesystem (this takes ~60 seconds)...
Setting disk label to: EXT FLASH
Formatted flash!
Flash chip successfully formatted with new empty filesystem!

Now you are ready to use the SPI Flash with FAT FS.

If you want to use the filesystem FAT, I advise you to use a SPIFlash with more than 1Mb of space.

Here is a simple sketch that creates a test.txt file with the string <num> -> testing 1, 2, 3. 100 times.

/*
 *  Add new txt file and add  the string <num> -> testing 1, 2, 3. 100 times
 *  than read the file and put on serial output.
 *  library Adafruit_SPIFlash and SdFat - AdafruitFork
 *
 *  by Mischianti Renzo <https://mischianti.org>
 *
 *  https://mischianti.org/
 *
 *	SPIFlash connected via SPI standard check wiring on the article
 *
 */

#include "SdFat.h"
#include "Adafruit_SPIFlash.h"

Adafruit_FlashTransport_SPI flashTransport(SS, SPI); // Set CS and SPI interface
Adafruit_SPIFlash flash(&flashTransport);

// file system object from SdFat
FatFileSystem fatfs;
File myFile;

void setup()
{
  Serial.begin(115200);
  while ( !Serial ) delay(100);   // wait for native usb

//  // Settings for the Winbond W25Q80BV (1 MiB SPI flash.)
//  // https://pdf1.alldatasheet.com/datasheet-pdf/view/555565/WINBOND/W25Q80BV.html
//  #define W25X80BV                                                            \
//    {                                                                            \
//      .total_size = (1UL << 20), /* 1 MiB */                                     \
//          .start_up_time_us = 5000, .manufacturer_id = 0xef,                     \
//      .memory_type = 0x40, .capacity = 0x14, .max_clock_speed_mhz = 104,         \
//      .quad_enable_bit_mask = 0x02, .has_sector_protection = false,              \
//      .supports_fast_read = true, .supports_qspi = true,                         \
//      .supports_qspi_writes = true, .write_status_register_split = false,        \
//      .single_status_byte = false, .is_fram = false,                             \
//    }
//
//
//  /// List of all possible flash devices used by Adafruit boards
//  static const SPIFlash_Device_t possible_devices[] = {
//      // Main devices used in current Adafruit products
//		  W25X80BV,
//  };

//if (flash.begin(possible_devices)) {
  if (flash.begin()) {
	  Serial.println(F("Device finded and supported!"));
  } else {
	  Serial.println(F("Problem to discover and configure device, check wiring also!"));
  }
  // Set 24Mhz SPI speed
  flashTransport.setClockSpeed(24000000, 24000000); // added to prevent speed problem

  pinMode(LED_BUILTIN, OUTPUT);
  flash.setIndicator(LED_BUILTIN, true);

  Serial.println("Adafruit Serial Flash read write example");
  Serial.print("JEDEC ID: "); Serial.println(flash.getJEDECID(), HEX);
  Serial.print("Flash size: "); Serial.println(flash.size());
  Serial.flush();

  Serial.println();
  // First call begin to mount the filesystem.  Check that it returns true
  // to make sure the filesystem was mounted.
  if (!fatfs.begin(&flash)) {
    Serial.println("Error, failed to mount newly formatted filesystem!");
    Serial.println("Was the flash chip formatted with the SdFat_format example?");
    while(1) yield();
  }
  Serial.println("Mounted filesystem!");
  Serial.println();

  // open the file. note that only one file can be open at a time,
  // so you have to close this one before opening another.
  myFile = fatfs.open("test.txt", FILE_WRITE);

  // if the file opened okay, write to it:
  if (myFile) {
    Serial.print("Writing to test.txt...");
    for (int i=0;i<100;i++){
    	myFile.print(i);
    	myFile.println(" -> testing 1, 2, 3.");
    }
    // close the file:
    myFile.close();
    Serial.println("done.");
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }

  // re-open the file for reading:
  myFile = fatfs.open("test.txt");
  if (myFile) {
    Serial.println("test.txt:");

    // read from the file until there's nothing else in it:
    while (myFile.available()) {
      Serial.write(myFile.read());
    }
    // close the file:
    myFile.close();
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }
}

void loop() {
  // nothing happens after setup
}

The result becomes like so:

Device finded and supported!
Adafruit Serial Flash Speed Test example
JEDEC ID: EF4017
Flash size: 8388608

Mounted filesystem!

Writing to test.txt...done.
test.txt:
0 -> testing 1, 2, 3.
[...]
99 -> testing 1, 2, 3.
ets Jun  8 2016 00:22:57

Now we can call some command to retrieve the filesystem info and then get the list of files with the ls command.

/*
 *  Retrieve basic core info of Flash SPI
 *  Get size and free size
 *
 *  than call ls command
 *
 *  library Adafruit_SPIFlash and SdFat - AdafruitFork
 *
 *  by Mischianti Renzo <https://mischianti.org>
 *
 *  https://mischianti.org/
 *
 *	SPIFlash connected via SPI standard check wiring on the article
 *
 */

#include "SdFat.h"
#include "Adafruit_SPIFlash.h"

Adafruit_FlashTransport_SPI flashTransport(SS, SPI); // Set CS and SPI interface
Adafruit_SPIFlash flash(&flashTransport);

// file system object from SdFat
FatFileSystem fatfs;

void setup()
{
  Serial.begin(115200);
  while ( !Serial ) delay(100);   // wait for native usb

//  // Settings for the Winbond W25Q80BV (1 MiB SPI flash.)
//  // https://pdf1.alldatasheet.com/datasheet-pdf/view/555565/WINBOND/W25Q80BV.html
//  #define W25X80BV                                                            \
//    {                                                                            \
//      .total_size = (1UL << 20), /* 1 MiB */                                     \
//          .start_up_time_us = 5000, .manufacturer_id = 0xef,                     \
//      .memory_type = 0x40, .capacity = 0x14, .max_clock_speed_mhz = 104,         \
//      .quad_enable_bit_mask = 0x02, .has_sector_protection = false,              \
//      .supports_fast_read = true, .supports_qspi = true,                         \
//      .supports_qspi_writes = true, .write_status_register_split = false,        \
//      .single_status_byte = false, .is_fram = false,                             \
//    }
//
//
//  /// List of all possible flash devices used by Adafruit boards
//  static const SPIFlash_Device_t possible_devices[] = {
//      // Main devices used in current Adafruit products
//		  W25X80BV,
//  };

//if (flash.begin(possible_devices)) {
  if (flash.begin()) {
	  Serial.println(F("Device finded and supported!"));
  } else {
	  Serial.println(F("Problem to discover and configure device, check wiring also!"));
  }
  // Set 24Mhz SPI speed
  flashTransport.setClockSpeed(24000000, 24000000); // added to prevent speed problem

  pinMode(LED_BUILTIN, OUTPUT);
  flash.setIndicator(LED_BUILTIN, true);

  Serial.println("Adafruit Serial Flash Speed Test example");
  Serial.print("JEDEC ID: "); Serial.println(flash.getJEDECID(), HEX);
  Serial.print("Flash size: "); Serial.println(flash.size());
  Serial.flush();

  Serial.println();
  // First call begin to mount the filesystem.  Check that it returns true
  // to make sure the filesystem was mounted.
  if (!fatfs.begin(&flash)) {
    Serial.println("Error, failed to mount newly formatted filesystem!");
    Serial.println("Was the flash chip formatted with the SdFat_format example?");
    while(1) yield();
  }
  Serial.println("Mounted filesystem!");
  Serial.println();

  Serial.print("Clusters:          ");
  Serial.println(fatfs.clusterCount());
  Serial.print("Blocks x Cluster:  ");
  Serial.println(fatfs.blocksPerCluster());

  Serial.print("Total Blocks:      ");
  Serial.println(fatfs.blocksPerCluster() * fatfs.clusterCount());
  Serial.println();

  // print the type and size of the first FAT-type volume
  uint32_t volumesize;
  Serial.print("Volume type is:    FAT");
  Serial.println(fatfs.fatType(), DEC);

  volumesize = fatfs.blocksPerCluster();    // clusters are collections of blocks
  volumesize *= fatfs.clusterCount();       // we'll have a lot of clusters

  volumesize /= 2;                           // SD card blocks are always 512 bytes (2 blocks are 1KB)
  Serial.print("Volume size (Kb):  ");
  Serial.println(volumesize);
  Serial.print("Volume size (Mb):  ");
  volumesize /= 1024;
  Serial.println(volumesize);

  Serial.println();
  // print the type and size of the first FAT-type volume
  volumesize = 0;
  volumesize = fatfs.blocksPerCluster();    // clusters are collections of blocks
  volumesize *= fatfs.freeClusterCount();       // we'll have a lot of clusters

  volumesize /= 2;                           // SD card blocks are always 512 bytes (2 blocks are 1KB)
  Serial.print("Free space size (Kb):  ");
  Serial.println(volumesize);
  Serial.print("Free space size (Mb):  ");
  volumesize /= 1024;
  Serial.println(volumesize);


  Serial.println("\nFiles found on the card (name, date and size in bytes): ");
  fatfs.rootDirStart();

  // list all files in the card with date and size
  fatfs.ls(LS_R | LS_DATE | LS_SIZE);
}

void loop(void) {
}

The result becomes

Device finded and supported!
Adafruit Serial Flash Speed Test example
JEDEC ID: EF4017
Flash size: 8388608

Mounted filesystem!

Clusters:          2043
Blocks x Cluster:  8
Total Blocks:      16344

Volume type is:    FAT12
Volume size (Kb):  8172
Volume size (Mb):  7

Free space size (Kb):  8164 
Free space size (Mb):  7

Files found on the card (name, date and size in bytes): 
2000-01-01 01:00:00       7170 test.txt

Thanks

Firmware and OTA update management
1. Firmware management
  1. ESP32: flash compiled firmware (.bin)
  2. ESP32: flash compiled firmware and filesystem (.bin) with GUI tools
2. OTA update with Arduino IDE
  1. ESP32 OTA update with Arduino IDE: filesystem, firmware, and password
3. OTA update with Web Browser
4. Self OTA uptate from HTTP server
5. Non-standard Firmware update
  1. ESP32 firmware and filesystem update from SD card
  2. ESP32 firmware and filesystem update with FTP client

Integrating LAN8720 with ESP32 for Ethernet Connectivity with plain (HTTP) and SSL (HTTPS)
Connecting the EByte E70 to ESP32 c3/s3 devices and a simple sketch example
ESP32-C3: pinout, specs and Arduino IDE configuration
Integrating W5500 with ESP32 Using Core 3: Native Ethernet Protocol Support with SSL and Other Features
Integrating LAN8720 with ESP32 Using Core 3: Native Ethernet Protocol Support with SSL and Other Features

Support forum

WeMos D1 mini (esp8266), specs and IDE configuration
WeMos D1 mini (esp8266), integrated SPIFFS Filesystem
WeMos D1 mini (esp8266), debug on secondary UART
WeMos D1 mini (esp8266), the three type of sleep mode to manage energy savings
WeMos D1 mini (esp8266), integrated LittleFS Filesystem
esp12 esp07 (esp8266): flash, pinout, specs and IDE configuration
Firmware and OTA update management
1. Firmware management
  1. esp8266: flash firmware binary (.bin) compiled and signed
  2. esp8266: flash firmware and filesystem binary (.bin) compiled with GUI tools
2. OTA update with Arduino IDE
  1. esp8266 OTA update with Arduino IDE: filesystem, signed and password
3. OTA update with Web Browser
4. Self OTA uptate from HTTP server
5. Non standard Firmware update
  1. esp8266 firmware and filesystem update from SD card
  2. esp8266 firmware and filesystem update with FTP client
esp32 and esp8266: FAT filesystem on external SPI flash memory
i2c esp8266: how to, network 5v, 3.3v, speed, and custom pins
[…]

Support forum

Spread the love

Tags: esp32 esp8266 Flash memories SPI Storage devices

Markus says:

17 February 2023 at 16:35

Hello Renzo,
thanks for another interesting guide.
Can the SPI Flash accessed by your FTP lib over (Wifi-) network?
http://mischianti.org/2021/07/01/simple-ftp-server-library-now-with-support-for-wio-terminal-and-sd/
Or are there any limitations?
Thanks!
Markus

Reply
- Renzo Mischianti says:
  
  17 February 2023 at 23:07
  
  Hi Markus,
  I add support for a lot of filesystems, you must check the list in the FtpServerKey.h
  Bye Renzo
  
  Reply
Dave says:

26 May 2023 at 18:21

This looks great, going to try it out this weekend! Thank you for the blog

Reply
Javier de la Vega says:

23 February 2024 at 16:15

Hello. Thank you for this blog and your great work.

I want to comment an issue I had to get the code working:

Using ESP32-S3 and custom SPI bus and Adafruit SPIFlash 4.3.4 (Current version at the moment of writting this comment) the program causes bootloop.

Taking a look at the library code, it seems that it ignores the Adafruit_FlashTransport_SPI passed in the Adafruit_SPIFlash’s constructor, in favor of Adafruit_FlashTransport_ESP32 wich is only intended the ESP32 onchip flash memory since Adafruit does not build any ESP32 product with external Flash chip.

When I modified this line
from
```
#if defined(ARDUINO_ARCH_ESP32) || defined(ARDUINO_ARCH_RP2040) 
```
to
```
#if defined(ARDUINO_ARCH_RP2040)
```
bypassing that condition, the code worked ok.

Im still searching for a definitive solution for this. since I don’t want to fork the library only for this tiny modification.

Did you have any problem like this?

Do you have any idea for a definitive solution?

Thanks!

Reply
- Renzo Mischianti says:
  
  24 February 2024 at 11:28
  
  Hi Javier,
  you touch a problem of esp32 variant check that I have in some libraries.
  The only define good for this is CONFIG_ARDUINO_VARIANT that have a value like
```
#define CONFIG_ARDUINO_VARIANT "esp32"
```
  or
```
#define CONFIG_ARDUINO_VARIANT "esp32s3"
```
  but It can be used only at runtime.
  In compile time the solution can be
  CONFIG_IDF_TARGET_ESP32
  So to select only ESP32 become
```
#if defined(ARDUINO_ARCH_RP2040) || define(CONFIG_IDF_TARGET_ESP32)
```
  to remove only the s3 variant become
```
#if defined(ARDUINO_ARCH_RP2040) || !define(CONFIG_IDF_TARGET_ESP32S3)
```
  Bye Renzo
  
  Reply