GitLab

A PSCI CPU_SUSPEND request to place a CPU in retention states at power levels
higher than the CPU power level is subject to the same state coordination as a
power down state. A CPU could implement multiple retention states at a
particular power level. When exiting WFI, the non-CPU power levels may be in a
different retention state to what was initially requested, therefore each CPU
should refresh its view of the states of all power levels.

Previously, a CPU re-used the state of the power levels when it entered the
retention state. This patch fixes this issue by ensuring that a CPU upon exit
from retention reads the state of each power level afresh.

Change-Id: I93b5f5065c63400c6fd2598dbaafac385748f989

This patch introduces the PSCI Library interface. The major changes
introduced are as follows:

* Earlier BL31 was responsible for Architectural initialization during cold
boot via bl31_arch_setup() whereas PSCI was responsible for the same during
warm boot. This functionality is now consolidated by the PSCI library
and it does Architectural initialization via psci_arch_setup() during both
cold and warm boots.

* Earlier the warm boot entry point was always `psci_entrypoint()`. This was
not flexible enough as a library interface. Now PSCI expects the runtime
firmware to provide the entry point via `psci_setup()`. A new function
`bl31_warm_entrypoint` is introduced in BL31 and the previous
`psci_entrypoint()` is deprecated.

* The `smc_helpers.h` is reorganized to separate the SMC Calling Convention
defines from the Trusted Firmware SMC helpers. The former is now in a new
header file `smcc.h` and the SMC helpers are moved to Architecture specific
header.

* The CPU context is used by PSCI for context initialization and
restoration after power down (PSCI Context). It is also used by BL31 for SMC
handling and context management during Normal-Secure world switch (SMC
Context). The `psci_smc_handler()` interface is redefined to not use SMC
helper macros thus enabling to decouple the PSCI context from EL3 runtime
firmware SMC context. This enables PSCI to be integrated with other runtime
firmware using a different SMC context.

NOTE: With this patch the architectural setup done in `bl31_arch_setup()`
is done as part of `psci_setup()` and hence `bl31_platform_setup()` will be
invoked prior to architectural setup. It is highly unlikely that the platform
setup will depend on architectural setup and cause any failure. Please be
be aware of this change in sequence.

Change-Id: I7f497a08d33be234bbb822c28146250cb20dab73

This patch moves the PSCI services and BL31 frameworks like context
management and per-cpu data into new library components `PSCI` and
`el3_runtime` respectively. This enables PSCI to be built independently from
BL31. A new `psci_lib.mk` makefile is introduced which adds the relevant
PSCI library sources and gets included by `bl31.mk`. Other changes which
are done as part of this patch are:

* The runtime services framework is now moved to the `common/` folder to
  enable reuse.
* The `asm_macros.S` and `assert_macros.S` helpers are moved to architecture
  specific folder.
* The `plat_psci_common.c` is moved from the `plat/common/aarch64/` folder
  to `plat/common` folder. The original file location now has a stub which
  just includes the file from new location to maintain platform compatibility.

Most of the changes wouldn't affect platform builds as they just involve
changes to the generic bl1.mk and bl31.mk makefiles.

NOTE: THE `plat_psci_common.c` FILE HAS MOVED LOCATION AND THE STUB FILE AT
THE ORIGINAL LOCATION IS NOW DEPRECATED. PLATFORMS SHOULD MODIFY THEIR
MAKEFILES TO INCLUDE THE FILE FROM THE NEW LOCATION.

Change-Id: I6bd87d5b59424995c6a65ef8076d4fda91ad5e86

This patch fixes some coding guideline warnings reported by the checkpatch
script. Only files related to upcoming feature development have been fixed.

Change-Id: I26fbce75c02ed62f00493ed6c106fe7c863ddbc5

This patch reworks type usage in generic code, drivers and ARM platform files
to make it more portable. The major changes done with respect to
type usage are as listed below:

* Use uintptr_t for storing address instead of uint64_t or unsigned long.
* Review usage of unsigned long as it can no longer be assumed to be 64 bit.
* Use u_register_t for register values whose width varies depending on
  whether AArch64 or AArch32.
* Use generic C types where-ever possible.

In addition to the above changes, this patch also modifies format specifiers
in print invocations so that they are AArch64/AArch32 agnostic. Only files
related to upcoming feature development have been reworked.

Change-Id: I9f8c78347c5a52ba7027ff389791f1dad63ee5f8

This patch adds following optional PSCI STAT functions:

- PSCI_STAT_RESIDENCY: This call returns the amount of time spent
  in power_state in microseconds, by the node represented by the
  `target_cpu` and the highest level of `power_state`.

- PSCI_STAT_COUNT: This call returns the number of times a
  `power_state` has been used by the node represented by the
  `target_cpu` and the highest power level of `power_state`.

These APIs provides residency statistics for power states that has
been used by the platform. They are implemented according to v1.0
of the PSCI specification.

By default this optional feature is disabled in the PSCI
implementation. To enable it, set the boolean flag
`ENABLE_PSCI_STAT` to 1. This also sets `ENABLE_PMF` to 1.

Change-Id: Ie62e9d37d6d416ccb1813acd7f616d1ddd3e8aff

When BL31 is compiled at `-O3` optimization level using Linaro GCC 4.9
AArch64 toolchain, it reports the following error:

```
services/std_svc/psci/psci_common.c: In function 'psci_do_state_coordination':
services/std_svc/psci/psci_common.c:220:27: error: array subscript is above
array bounds [-Werror=array-bounds]
  psci_req_local_pwr_states[pwrlvl - 1][cpu_idx] = req_pwr_state;
                           ^
```

This error is a false positive and this patch resolves the error by asserting
the array bounds in `psci_do_state_coordination()`.

Fixes ARM-software/tf-issues#347

Change-Id: I3584ed7b2e28faf455b082cb3281d6e1d11d6495

Migrate all direct usage of __attribute__ to usage of their
corresponding macros from cdefs.h.
e.g.:
 - __attribute__((unused)) -> __unused
Signed-off-by: Soren Brinkmann <soren.brinkmann@xilinx.com>

This patch unifies the bakery lock api's across coherent and normal
memory implementation of locks by using same data type `bakery_lock_t`
and similar arguments to functions.

A separate section `bakery_lock` has been created and used to allocate
memory for bakery locks using `DEFINE_BAKERY_LOCK`. When locks are
allocated in normal memory, each lock for a core has to spread
across multiple cache lines. By using the total size allocated in a
separate cache line for a single core at compile time, the memory for
other core locks is allocated at link time by multiplying the single
core locks size with (PLATFORM_CORE_COUNT - 1). The normal memory lock
algorithm now uses lock address instead of the `id` in the per_cpu_data.
For locks allocated in coherent memory, it moves locks from
tzfw_coherent_memory to bakery_lock section.

The bakery locks are allocated as part of bss or in coherent memory
depending on usage of coherent memory. Both these regions are
initialised to zero as part of run_time_init before locks are used.
Hence, bakery_lock_init() is made an empty function as the lock memory
is already initialised to zero.

The above design lead to the removal of psci bakery locks from
non_cpu_power_pd_node to psci_locks.

NOTE: THE BAKERY LOCK API WHEN USE_COHERENT_MEM IS NOT SET HAS CHANGED.
THIS IS A BREAKING CHANGE FOR ALL PLATFORM PORTS THAT ALLOCATE BAKERY
LOCKS IN NORMAL MEMORY.

Change-Id: Ic3751c0066b8032dcbf9d88f1d4dc73d15f61d8b

This patch reworks the PSCI generic implementation to conform to ARM
Trusted Firmware coding guidelines as described here:
https://github.com/ARM-software/arm-trusted-firmware/wiki

This patch also reviews the use of signed data types within PSCI
Generic code and replaces them with their unsigned counterparts wherever
they are not appropriate. The PSCI_INVALID_DATA macro which was defined
to -1 is now replaced with PSCI_INVALID_PWR_LVL macro which is defined
to PLAT_MAX_PWR_LVL + 1.

Change-Id: Iaea422d0e46fc314e0b173c2b4c16e0d56b2515a

As per PSCI1.0 specification, the error code to be returned when an invalid
non secure entrypoint address is specified by the PSCI client for CPU_SUSPEND,
CPU_ON or SYSTEM_SUSPEND must be PSCI_E_INVALID_ADDRESS. The current PSCI
implementation returned PSCI_E_INVAL_PARAMS. This patch rectifies this error
and also implements a common helper function to validate the entrypoint
information to be used across these PSCI API implementations.

Change-Id: I52d697d236c8bf0cd3297da4008c8e8c2399b170

This commit does the switch to the new PSCI framework implementation replacing
the existing files in PSCI folder with the ones in PSCI1.0 folder. The
corresponding makefiles are modified as required for the new implementation.
The platform.h header file is also is switched to the new one
as required by the new frameworks. The build flag ENABLE_PLAT_COMPAT defaults
to 1 to enable compatibility layer which let the existing platform ports to
continue to build and run with minimal changes.

The default weak implementation of platform_get_core_pos() is now removed from
platform_helpers.S and is provided by the compatibility layer.

Note: The Secure Payloads and their dispatchers still use the old platform
and framework APIs and hence it is expected that the ENABLE_PLAT_COMPAT build
flag will remain enabled in subsequent patch. The compatibility for SPDs using
the older APIs on platforms migrated to the new APIs will be added in the
following patch.

Change-Id: I18c51b3a085b564aa05fdd98d11c9f3335712719

This patch adds support for SYSTEM_SUSPEND API as mentioned in the PSCI 1.0
specification. This API, on being invoked on the last running core on a
supported platform, will put the system into a low power mode with memory
retention.

The psci_afflvl_suspend() internal API has been reused as most of the actions
to suspend a system are the same as invoking the PSCI CPU_SUSPEND API with the
target affinity level as 'system'. This API needs the 'power state' parameter
for the target low power state. This parameter is not passed by the caller of
the SYSTEM_SUSPEND API. Hence, the platform needs to implement the
get_sys_suspend_power_state() platform function to provide this information.
Also, the platform also needs to add support for suspending the system to the
existing 'plat_pm_ops' functions: affinst_suspend() and
affinst_suspend_finish().

Change-Id: Ib6bf10809cb4e9b92f463755608889aedd83cef5

mpidr_set_aff_inst() is left shifting an int constant and an
unsigned char value to construct an MPIDR. For affinity level 3 a
shift of 32 would result in shifting out of the 32-bit type and
have no effect on the MPIDR.

These values need to be extended to unsigned long before shifting
to ensure correct results for affinity level 3.

Change-Id: I1ef40afea535f14cfd820c347a065a228e8f4536

All coding style violations have been fixed in a previous patch and
since then, each individual patch has been checked in this regard.
However, the latest version of the checkpatch.pl script from the Linux
kernel is more advanced and it is able to flag new errors in the
Trusted Firmware codebase. This patch fixes them.

Change-Id: I1f332f2440984be85d36b231bb83260368987077

This patch removes the plat_get_max_afflvl() platform API
and instead replaces it with a platform macro PLATFORM_MAX_AFFLVL.
This is done because the maximum affinity level for a platform
is a static value and it is more efficient for it to be defined
as a platform macro.

NOTE: PLATFORM PORTS NEED TO BE UPDATED ON MERGE OF THIS COMMIT

Fixes ARM-Software/tf-issues#265

Change-Id: I31d89b30c2ccda30d28271154d869060d50df7bf

This patch implements the PSCI_FEATURES function which is a mandatory
API in the PSCI 1.0 specification. A capability variable is
constructed during initialization by examining the plat_pm_ops and
spd_pm_ops exported by the platform and the Secure Payload Dispatcher.
This is used by the PSCI FEATURES function to determine which
PSCI APIs are supported by the platform.

Change-Id: I147ffc1bd5d90b469bd3cc4bbe0a20e95c247df7

This patch reworks the PSCI MIGRATE, MIGRATE_INFO_TYPE and
MIGRATE_INFO_UP_CPU support for Trusted Firmware. The
implementation does the appropriate validation of parameters
and invokes the appropriate hook exported by the SPD.

The TSP is a MP Trusted OS. Hence the ability to actually
migrate a Trusted OS has not been implemented. The
corresponding function is not populated in the spd_pm_hooks
structure for the TSPD.

The `spd_pm_ops_t` has undergone changes with this patch.
SPD PORTS MAY NEED TO BE UPDATED.

Fixes ARM-software/tf-issues#249

Change-Id: Iabd87521bf7c530a5e4506b6d3bfd4f1bf87604f

This patch allows the platform to validate the power_state and
entrypoint information from the normal world early on in PSCI
calls so that we can return the error safely. New optional
pm_ops hooks `validate_power_state` and `validate_ns_entrypoint`
are introduced to do this.

As a result of these changes, all the other pm_ops handlers except
the PSCI_ON handler are expected to be successful. Also, the PSCI
implementation will now assert if a PSCI API is invoked without the
corresponding pm_ops handler being registered by the platform.

NOTE : PLATFORM PORTS WILL BREAK ON MERGE OF THIS COMMIT. The
pm hooks have 2 additional optional callbacks and the return type
of the other hooks have changed.

Fixes ARM-Software/tf-issues#229

Change-Id: I036bc0cff2349187c7b8b687b9ee0620aa7e24dc

This patch replaces the internal psci_save_ns_entry() API with a
psci_get_ns_ep_info() API. The new function splits the work done by the
previous one such that it populates and returns an 'entry_point_info_t'
structure with the information to enter the normal world upon completion
of the CPU_SUSPEND or CPU_ON call. This information is used to populate
the non-secure context structure separately.

This allows the new internal API `psci_get_ns_ep_info` to return error
and enable the code to return safely.

Change-Id: Ifd87430a4a3168eac0ebac712f59c93cbad1b231

This patch extends the build option `USE_COHERENT_MEMORY` to
conditionally remove coherent memory from the memory maps of
all boot loader stages. The patch also adds necessary
documentation for coherent memory removal in firmware-design,
porting and user guides.

Fixes ARM-Software/tf-issues#106

Change-Id: I260e8768c6a5c2efc402f5804a80657d8ce38773

This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.

As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.

Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0

This patch fixes the assertion failure when CPU_SUSPEND is invoked with
an affinity level higher than supported by the platform by adding suitable
checks for affinity level within `psci_cpu_suspend`. Also added suitable
bound checks within `psci_aff_map_get_idx` to prevent indexing beyond array
limits.

Fixes ARM-software/tf-issues#260

Change-Id: I04b75c49729e6c6d1983add590f60146c8fc3630

This patch implements the following cleanups in PSCI generic code:

1. It reworks the affinity level specific handlers in the PSCI implementation
   such that.

   a. Usage of the 'rc' local variable is restricted to only where it is
      absolutely needed

   b. 'plat_state' local variable is defined only when a direct invocation of
      plat_get_phys_state() does not suffice.

   c. If a platform handler is not registered then the level specific handler
      returns early.

2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
   arrays of the type instead of using it in function prototypes as well.

3. It removes dangling declarations of __psci_cpu_off() and
   __psci_cpu_suspend(). The definitions of these functions were removed in
   earlier patches.

Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

This patch pulls out state management from the affinity level specific handlers
into the top level functions specific to the operation
i.e. psci_afflvl_suspend(), psci_afflvl_on() etc.

In the power down path this patch will allow an affinity instance at level X to
determine the state that an affinity instance at level X+1 will enter before the
level specific handlers are called. This will be useful to determine whether a
CPU is the last in the cluster during a suspend/off request and so on.

Similarly, in the power up path this patch will allow an affinity instance at
level X to determine the state that an affinity instance at level X+1 has
emerged from, even after the level specific handlers have been called. This will
be useful in determining whether a CPU is the first in the cluster during a
on/resume request and so on.

As before, while powering down, state is updated before the level specific
handlers are invoked so that they can perform actions based upon their target
state. While powering up, state is updated after the level specific handlers have
been invoked so that they can perform actions based upon the state they emerged
from.

Change-Id: I40fe64cb61bb096c66f88f6d493a1931243cfd37

This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.

The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.

The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.

The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.

Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf

This patch adds support for SYSTEM_OFF and SYSTEM_RESET PSCI
operations. A platform should export handlers to complete the
requested operation. The FVP port exports fvp_system_off() and
fvp_system_reset() as an example.

If the SPD provides a power management hook for system off and
system reset, then the SPD is notified about the corresponding
operation so it can do some bookkeeping. The TSPD exports
tspd_system_off() and tspd_system_reset() for that purpose.

Versatile Express shutdown and reset methods have been removed
from the FDT as new PSCI sys_poweroff and sys_reset services
have been added. For those kernels that do not support yet these
PSCI services (i.e. GICv3 kernel), the original dtsi files have
been renamed to *-no_psci.dtsi.

Fixes ARM-software/tf-issues#218

Change-Id: Ic8a3bf801db979099ab7029162af041c4e8330c8

This patch removes the allocation of memory for coherent stacks, associated
accessor function and some dead code which called the accessor function. It also
updates the porting guide to remove the concept and the motivation behind using
stacks allocated in coherent memory.

Fixes ARM-software/tf-issues#198

Change-Id: I00ff9a04f693a03df3627ba39727e3497263fc38

Many of the interfaces internal to PSCI pass the current CPU
MPIDR_EL1 value from function to function. This is not required,
and with inline access to the system registers is less efficient
than requiring the code to read that register whenever required.

This patch remove the mpidr parameter from the affected interfaces
and reduces code in FVP BL3-1 size by 160 bytes.

Change-Id: I16120a7c6944de37232016d7e109976540775602

Exclude stdlib files because they do not follow kernel code style.

Fixes ARM-software/tf-issues#73

Change-Id: I4cfafa38ab436f5ab22c277cb38f884346a267ab

The bakery lock code currently expects the calling code to pass
the MPIDR_EL1 of the current CPU.

This is not always done correctly. Also the change to provide
inline access to system registers makes it more efficient for the
bakery lock code to obtain the MPIDR_EL1 directly.

This change removes the mpidr parameter from the bakery lock
interface, and results in a code reduction of 160 bytes for the
ARM FVP port.

Fixes ARM-software/tf-issues#213

Change-Id: I7ec7bd117bcc9794a0d948990fcf3336a367d543

psci_suspend_context is an array of cache-line aligned structures
containing the single power_state integer per cpu. This array is
the only structure indexed by the aff_map_node.data integer.

This patch saves 2KB of BL3-1 memory by placing the CPU
power_state value directly in the aff_map_node structure. As a
result, this value is now never cached and the cache clean when
writing the value is no longer required.

Fixes ARM-software/tf-issues#195

Change-Id: Ib4c70c8f79eed295ea541e7827977a588a19ef9b

Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
*  The first uses entry_point_info to initialize the relevant
   cpu_context for first entry into a lower exception level on a CPU
*  The second populates the EL1 and EL2 system registers as needed
   from the cpu_context to ensure correct entry into the lower EL

This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).

In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.

Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f

All callers of cm_get_context() pass the calling CPU MPIDR to the
function. Providing a specialised version for the current
CPU results in a reduction in code size and better readability.

The current function has been renamed to cm_get_context_by_mpidr()
and the existing name is now used for the current-CPU version.

The same treatment has been done to cm_set_context(), although
only both forms are used at present in the PSCI and TSPD code.

Change-Id: I91cb0c2f7bfcb950a045dbd9ff7595751c0c0ffb

Previously, platform.h contained many declarations and definitions
used for different purposes. This file has been split so that:

* Platform definitions used by common code that must be defined
  by the platform are now in platform_def.h. The exact include
  path is exported through $PLAT_INCLUDES in the platform makefile.

* Platform definitions specific to the FVP platform are now in
  /plat/fvp/fvp_def.h.

* Platform API declarations specific to the FVP platform are now
  in /plat/fvp/fvp_private.h.

* The remaining platform API declarations that must be ported by
  each platform are still in platform.h but this file has been
  moved to /include/plat/common since this can be shared by all
  platforms.

Change-Id: Ieb3bb22fbab3ee8027413c6b39a783534aee474a

Some data variables were declared but not used. These have been
removed.

Change-Id: I038632af3c32d88984cd25b886c43ff763269bf9

This patch introduces macros (SPSR_64 and SPSR_32) to
create a SPSR for both aarch32 and aarch64 execution
states. These macros allow the user to set fields
in the SPSR depending upon its format.
The make_spsr() function which did not allow
manipulation of all the fields in the aarch32 SPSR
has been replaced by these new macros.

Change-Id: I9425dda0923e8d5f03d03ddb8fa0e28392c4c61e

Update code base to remove variables from the .data section,
mainly by using const static data where possible and adding
the const specifier as required. Most changes are to the IO
subsystem, including the framework APIs. The FVP power
management code is also affected.

Delay initialization of the global static variable,
next_image_type in bl31_main.c, until it is realy needed.
Doing this moves the variable from the .data to the .bss
section.

Also review the IO interface for inconsistencies, using
uintptr_t where possible instead of void *. Remove the
io_handle and io_dev_handle typedefs, which were
unnecessary, replacing instances with uintptr_t.

Fixes ARM-software/tf-issues#107.

Change-Id: I085a62197c82410b566e4698e5590063563ed304

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e