Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EEPROM library on logging principle (from STM32) #254

Open
GeorgeBobrov opened this issue Mar 21, 2023 · 10 comments
Open

EEPROM library on logging principle (from STM32) #254

GeorgeBobrov opened this issue Mar 21, 2023 · 10 comments

Comments

@GeorgeBobrov
Copy link

Hello dbuezas!
On LGT8F328p, EEPROM is emulated using space from the main Flash memory. The emulation uses a technic of page swapping: each time we write to the emulated EEPROM, the device erases the new page, copies the old data from the old page and updates the new data on the fly on the new page, then it swaps the old and new page.
The STM32 also does not have an EEPROM and is also emulated using the main Flash memory space. However, it uses a much more economical logging principle in terms of the number of page erasures.
The principle is described in the document http://www.st.com/web/en/resource/technical/document/application_note/CD00165693.pdf.
In short: all changes are written to the log, and only when the log takes up a full page (1KB), the compressed log
is copied to the second page (only the latest actual values are copied).
The actual library code is here:
https://github.com/rogerclarkmelbourne/Arduino_STM32/tree/master/STM32F1/libraries/EEPROM
It would be nice if this principle was also ported to LGT8F328p (perhaps as a second version of the EEPROM library).
This will greatly reduce the number of Flash memory overwrites, which is already limited. In addition, such porting does not look too complicated.
What do you think about this? Do you want to try this task?

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Mar 21, 2023

If i remember correctly, the page-swap algorithm of the LGT8 is implemented in hardware.
A software write-buffer / log could be implemented, but there is only 2k of RAM on the LGT8. That's not a lot of room for a log / buffer.

Though, you can implement a write-buffer and use the serial-write functions to write it all at once using void lgt_eeprom_writeSWM( uint16_t address, uint32_t *pData, uint16_t length ). (The buffer has to be 32 bits, because that's the actual size of the flash memory cells.)

@GeorgeBobrov
Copy link
Author

I have read the documentation. Indeed, on LGT8F328p, the emulation of EEPROM uses a technic of page swapping at the hardware level. This mode is enabled in the ECCR register. The emulation mode is called E2PROM.
However, as far as I understand, there is a direct access mode to Flash memory, without E2PROM and page swapping, and this mode is used, for example, by the bootloader to update the firmware.
Perhaps, in this case, it is possible to implement the EEPROM emulation algorithm on raw Flash memory, on its free section, with E2PROM turned off, exactly as it is done in the library for STM32.

@SuperUserNameMan
Copy link
Contributor

I don't remember if it is possible to write directly to the flash once outside the bootloader :-/
Is it ?

Also, I have not read the STM32 pdf you provided yet, because I don't understand the benefit of storing a log in RAM given the fact that the LGT8 only has 2kB of RAM :

  • If the log requires more than 1kB of RAM to write a 1kB of Flash, there would be not enough RAM left free for other usage. :-/

What are the minimal requirements of this algorithm ?

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Mar 22, 2023

PS : it is possible to write to the flash without triggering the swap using the SWM functions and macros (SWM = Serial Write Mode).
However, at the minimum, it requires to write 4 bytes aligned on 32bits (because cells of the flash are 32bits).

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Mar 22, 2023

PS2 : also note that the hardware swap is not triggered if you write to a cell that contains 0xFFffFFff (it's blank).

@GeorgeBobrov
Copy link
Author

The algorithm from STM does not use RAM. It creates a log directly in Flash memory, appending it to the same Flash page without rewriting. And only when the log fills one page, the algorithm rewrites the data to another page, but not all, only the last actual values. To understand the meaning, you still need to read the document.

The point of this is that the number of Flash page rewrites is reduced by a factor of thousands. However, there is a drawback - this algorithm is effective only when a relatively small amount of settings is stored in the emulated EEPROM, much less than the full size of the emulated EEPROM. However, that is what usually happens.

@SuperUserNameMan
Copy link
Contributor

Maybe you could try implementing it ?

The SWM functions and macro I mentioned would allow you to write the log into blank pages without triggering hardware page swaps.

Given the fact that the cells of the flash are 32bits wide, they are large enough to store a tiny struct containing a value, an address and a flag for each log entry (if this is the approach you choose).

Also, keep in mind that the last 4 bytes of each 1k pages are reserved to the hardware page-swap algorithm.
Which mean, only the first 1020 bytes (or 255 cells) are available consecutively in each page.

@GeorgeBobrov
Copy link
Author

Porting itself will not be difficult if it is feasible in principle. And whether it is feasible or not depends on one fact: whether or not the LGT8F328 has the ability to directly access Flash memory pages, including arbitrary writing and erasing. There is definitely such an opportunity in the bootloader, the question is whether we can do this from the user program. If so, then the sequence of actions could be as follows:

  1. Disabling the hardware EEPROM emulation mechanism (you can disable it in the ECCR register)
  2. Using the same pages of Flash memory at the same addresses that were used in the hardware EEPROM emulation mode, but we need to use direct access to this memory (the page swapping mechanism must be disabled, since it will not allow you to manually transfer data from one page to another)
  3. All mechanisms for actually creating a log in Flash memory, saving data there, reading data from there, rewriting the log to another page when the current one is filled, have already been implemented in the library https://github.com/rogerclarkmelbourne/Arduino_STM32/tree/master/STM32F1/libraries/EEPROM in the EEPROM.cpp file
  4. The EEPROM.cpp file is written in a fairly high-level style, it does not work directly with Flash memory, but calls functions from the flash_stm32.c file. There are located the low-level functions for working with Flash memory:
    FLASH_ErasePage, FLASH_ProgramHalfWord, FLASH_WaitForLastOperation, FLASH_GetStatus
  5. In general, all the functions that need to be ported are described in the flash_stm32.h file, and there are only 6 of them.

The main thing, going back to the beginning, is the question whether direct access to Flash memory possible from a user program.
Unfortunately, I don't have enough low-level programming experience to answer this question.

@LaZsolt
Copy link
Collaborator

LaZsolt commented Mar 24, 2023

Basically pgm_read_byte() is the direct access.

But LGT8Fx capable read the FLASH like RAM from address 0x4000. The complier can't produce an optimal code when acessing this address range.

@dbuezas
Copy link
Owner

dbuezas commented Mar 30, 2023

This PR may be relevant in this discussion: #261
Particularly because it may be incompatible with the ideas discussed in this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants