GitLab

During cold boot, the initial translation tables are created with data
caches disabled, so all modifications go to memory directly. After the
MMU is enabled and data cache is enabled, any modification to the tables
goes to data cache, and eventually may get flushed to memory.

If CPU0 modifies the tables while CPU1 is off, CPU0 will have the
modified tables in its data cache. When CPU1 is powered on, the MMU is
enabled, then it enables coherency, and then it enables the data cache.
Until this is done, CPU1 isn't in coherency, and the translation tables
it sees can be outdated if CPU0 still has some modified entries in its
data cache.

This can be a problem in some cases. For example, the warm boot code
uses only the tables mapped during cold boot, which don't normally
change. However, if they are modified (and a RO page is made RW, or a XN
page is made executable) the CPU will see the old attributes and crash
when it tries to access it.

This doesn't happen in systems with HW_ASSISTED_COHERENCY or
WARMBOOT_ENABLE_DCACHE_EARLY. In these systems, the data cache is
enabled at the same time as the MMU. As soon as this happens, the CPU is
in coherency.

There was an attempt of a fix in psci_helpers.S, but it didn't solve the
problem. That code has been deleted. The code was introduced in commit
<26441030

> ("Invalidate TLB entries during warm boot").

Now, during a map or unmap operation, the memory associated to each
modified table is flushed. Traversing a table will also flush it's
memory, as there is no way to tell in the current implementation if the
table that has been traversed has also been modified.

Change-Id: I4b520bca27502f1018878061bc5fb82af740bb92
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

During the warm boot sequence:

1. The MMU is enabled with the data cache disabled. The MMU table walker
   is set up to access the translation tables as in cacheable memory,
   but its accesses are non-cacheable because SCTLR_EL3.C controls them
   as well.
2. The interconnect is set up and the CPU enters coherency with the
   rest of the system.
3. The data cache is enabled.

If the support for dynamic translation tables is enabled and another CPU
makes changes to a region, the changes may only be present in the data
cache, not in RAM. The CPU that is booting isn't in coherency with the
rest of the system, so the table walker of that CPU isn't either. This
means that it may read old entries from RAM and it may have invalid TLB
entries corresponding to the dynamic mappings.

This is not a problem for the boot code because the mapping is 1:1 and
the regions are static. However, the code that runs after the boot
sequence may need to access the dynamically mapped regions.

This patch invalidates all TLBs during warm boot when the dynamic
translation tables support is enabled to prevent this problem.

Change-Id: I80264802dc0aa1cb3edd77d0b66b91db6961af3d
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/



Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Various CPU drivers in ARM Trusted Firmware register functions to handle
power-down operations. At present, separate functions are registered to
power down individual cores and clusters.

This scheme operates on the basis of core and cluster, and doesn't cater
for extending the hierarchy for power-down operations. For example,
future CPUs might support multiple threads which might need powering
down individually.

This patch therefore reworks the CPU operations framework to allow for
registering power down handlers on specific level basis. Henceforth:

  - Generic code invokes CPU power down operations by the level
    required.

  - CPU drivers explicitly mention CPU_NO_RESET_FUNC when the CPU has no
    reset function.

  - CPU drivers register power down handlers as a list: a mandatory
    handler for level 0, and optional handlers for higher levels.

All existing CPU drivers are adapted to the new CPU operations framework
without needing any functional changes within.

Also update firmware design guide.

Change-Id: I1826842d37a9e60a9e85fdcee7b4b8f6bc1ad043
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

The AArch32 Procedure call Standard mandates that the stack must be aligned
to 8 byte boundary at external interfaces. This patch does the required
changes.

This problem was detected when a crash was encountered in
`psci_print_power_domain_map()` while printing 64 bit values. Aligning
the stack to 8 byte boundary resolved the problem.

Fixes ARM-Software/tf-issues#437

Change-Id: I517bd8203601bb88e9311bd36d477fb7b3efb292
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

There are many instances in ARM Trusted Firmware where control is
transferred to functions from which return isn't expected. Such jumps
are made using 'bl' instruction to provide the callee with the location
from which it was jumped to. Additionally, debuggers infer the caller by
examining where 'lr' register points to. If a 'bl' of the nature
described above falls at the end of an assembly function, 'lr' will be
left pointing to a location outside of the function range. This misleads
the debugger back trace.

This patch defines a 'no_ret' macro to be used when jumping to functions
from which return isn't expected. The macro ensures to use 'bl'
instruction for the jump, and also, for debug builds, places a 'nop'
instruction immediately thereafter (unless instructed otherwise) so as
to leave 'lr' pointing within the function range.

Change-Id: Ib34c69fc09197cfd57bc06e147cc8252910e01b0
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

This patch adds AArch32 support to PSCI library, as follows :

* The `psci_helpers.S` is implemented for AArch32.

* AArch32 version of internal helper function `psci_get_ns_ep_info()` is
  defined.

* The PSCI Library is responsible for the Non Secure context initialization.
  Hence a library interface `psci_prepare_next_non_secure_ctx()` is introduced
  to enable EL3 runtime firmware to initialize the non secure context without
  invoking context management library APIs.

Change-Id: I25595b0cc2dbfdf39dbf7c589b875cba33317b9d

This patch introduces the PSCI Library interface. The major changes
introduced are as follows:

* Earlier BL31 was responsible for Architectural initialization during cold
boot via bl31_arch_setup() whereas PSCI was responsible for the same during
warm boot. This functionality is now consolidated by the PSCI library
and it does Architectural initialization via psci_arch_setup() during both
cold and warm boots.

* Earlier the warm boot entry point was always `psci_entrypoint()`. This was
not flexible enough as a library interface. Now PSCI expects the runtime
firmware to provide the entry point via `psci_setup()`. A new function
`bl31_warm_entrypoint` is introduced in BL31 and the previous
`psci_entrypoint()` is deprecated.

* The `smc_helpers.h` is reorganized to separate the SMC Calling Convention
defines from the Trusted Firmware SMC helpers. The former is now in a new
header file `smcc.h` and the SMC helpers are moved to Architecture specific
header.

* The CPU context is used by PSCI for context initialization and
restoration after power down (PSCI Context). It is also used by BL31 for SMC
handling and context management during Normal-Secure world switch (SMC
Context). The `psci_smc_handler()` interface is redefined to not use SMC
helper macros thus enabling to decouple the PSCI context from EL3 runtime
firmware SMC context. This enables PSCI to be integrated with other runtime
firmware using a different SMC context.

NOTE: With this patch the architectural setup done in `bl31_arch_setup()`
is done as part of `psci_setup()` and hence `bl31_platform_setup()` will be
invoked prior to architectural setup. It is highly unlikely that the platform
setup will depend on architectural setup and cause any failure. Please be
be aware of this change in sequence.

Change-Id: I7f497a08d33be234bbb822c28146250cb20dab73

This patch moves the PSCI services and BL31 frameworks like context
management and per-cpu data into new library components `PSCI` and
`el3_runtime` respectively. This enables PSCI to be built independently from
BL31. A new `psci_lib.mk` makefile is introduced which adds the relevant
PSCI library sources and gets included by `bl31.mk`. Other changes which
are done as part of this patch are:

* The runtime services framework is now moved to the `common/` folder to
  enable reuse.
* The `asm_macros.S` and `assert_macros.S` helpers are moved to architecture
  specific folder.
* The `plat_psci_common.c` is moved from the `plat/common/aarch64/` folder
  to `plat/common` folder. The original file location now has a stub which
  just includes the file from new location to maintain platform compatibility.

Most of the changes wouldn't affect platform builds as they just involve
changes to the generic bl1.mk and bl31.mk makefiles.

NOTE: THE `plat_psci_common.c` FILE HAS MOVED LOCATION AND THE STUB FILE AT
THE ORIGINAL LOCATION IS NOW DEPRECATED. PLATFORMS SHOULD MODIFY THEIR
MAKEFILES TO INCLUDE THE FILE FROM THE NEW LOCATION.

Change-Id: I6bd87d5b59424995c6a65ef8076d4fda91ad5e86

This patch reworks the PSCI generic implementation to conform to ARM
Trusted Firmware coding guidelines as described here:
https://github.com/ARM-software/arm-trusted-firmware/wiki

This patch also reviews the use of signed data types within PSCI
Generic code and replaces them with their unsigned counterparts wherever
they are not appropriate. The PSCI_INVALID_DATA macro which was defined
to -1 is now replaced with PSCI_INVALID_PWR_LVL macro which is defined
to PLAT_MAX_PWR_LVL + 1.

Change-Id: Iaea422d0e46fc314e0b173c2b4c16e0d56b2515a

This commit does the switch to the new PSCI framework implementation replacing
the existing files in PSCI folder with the ones in PSCI1.0 folder. The
corresponding makefiles are modified as required for the new implementation.
The platform.h header file is also is switched to the new one
as required by the new frameworks. The build flag ENABLE_PLAT_COMPAT defaults
to 1 to enable compatibility layer which let the existing platform ports to
continue to build and run with minimal changes.

The default weak implementation of platform_get_core_pos() is now removed from
platform_helpers.S and is provided by the compatibility layer.

Note: The Secure Payloads and their dispatchers still use the old platform
and framework APIs and hence it is expected that the ENABLE_PLAT_COMPAT build
flag will remain enabled in subsequent patch. The compatibility for SPDs using
the older APIs on platforms migrated to the new APIs will be added in the
following patch.

Change-Id: I18c51b3a085b564aa05fdd98d11c9f3335712719

In order for the symbol table in the ELF file to contain the size of
functions written in assembly, it is necessary to report it to the
assembler using the .size directive.

To fulfil the above requirements, this patch introduces an 'endfunc'
macro which contains the .endfunc and .size directives. It also adds
a .func directive to the 'func' assembler macro.

The .func/.endfunc have been used so the assembler can fail if
endfunc is omitted.

Fixes ARM-Software/tf-issues#295

Change-Id: If8cb331b03d7f38fe7e3694d4de26f1075b278fc
Signed-off-by: Kévin Petit <kevin.petit@arm.com>

This patch adds CPU core and cluster power down sequences to the CPU specific
operations framework introduced in a earlier patch. Cortex-A53, Cortex-A57 and
generic AEM sequences have been added. The latter is suitable for the
Foundation and Base AEM FVPs. A pointer to each CPU's operations structure is
saved in the per-cpu data so that it can be easily accessed during power down
seqeunces.

An optional platform API has been introduced to allow a platform to disable the
Accelerator Coherency Port (ACP) during a cluster power down sequence. The weak
definition of this function (plat_disable_acp()) does not take any action. It
should be overriden with a strong definition if the ACP is present on a
platform.

Change-Id: I8d09bd40d2f528a28d2d3f19b77101178778685d

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

This patch uses stacks allocated in normal memory to enable the MMU early in the
warm boot path thus removing the dependency on stacks allocated in coherent
memory. Necessary cache and stack maintenance is performed when a cpu is being
powered down and up. This avoids any coherency issues that can arise from
reading speculatively fetched stale stack memory from another CPUs cache. These
changes affect the warm boot path in both BL3-1 and BL3-2.

The EL3 system registers responsible for preserving the MMU state are not saved
and restored any longer. Static values are used to program these system
registers when a cpu is powered on or resumed from suspend.

Change-Id: I8357e2eb5eb6c5f448492c5094b82b8927603784