GitLab

This patch optimizes the invocation of the platform power management hooks for
ON, OFF and SUSPEND such that they are called only for the highest affinity
level which will be powered off/on. Earlier, the hooks were being invoked for
all the intermediate levels as well.

This patch requires that the platforms migrate to the new semantics of the PM
hooks.  It also removes the `state` parameter from the pm hooks as the `afflvl`
parameter now indicates the highest affinity level for which power management
operations are required.

Change-Id: I57c87931d8a2723aeade14acc710e5b78ac41732

This patch creates a copy of the existing PSCI files and related psci.h and
platform.h header files in a new `PSCI1.0` directory. The changes for the
new PSCI power domain topology and extended state-ID frameworks will be
added incrementally to these files. This incremental approach will
aid in review and in understanding the changes better. Once all the
changes have been introduced, these files will replace the existing PSCI
files.

Change-Id: Ibb8a52e265daa4204e34829ed050bddd7e3316ff

This patch adds support for SYSTEM_SUSPEND API as mentioned in the PSCI 1.0
specification. This API, on being invoked on the last running core on a
supported platform, will put the system into a low power mode with memory
retention.

The psci_afflvl_suspend() internal API has been reused as most of the actions
to suspend a system are the same as invoking the PSCI CPU_SUSPEND API with the
target affinity level as 'system'. This API needs the 'power state' parameter
for the target low power state. This parameter is not passed by the caller of
the SYSTEM_SUSPEND API. Hence, the platform needs to implement the
get_sys_suspend_power_state() platform function to provide this information.
Also, the platform also needs to add support for suspending the system to the
existing 'plat_pm_ops' functions: affinst_suspend() and
affinst_suspend_finish().

Change-Id: Ib6bf10809cb4e9b92f463755608889aedd83cef5

mpidr_set_aff_inst() is left shifting an int constant and an
unsigned char value to construct an MPIDR. For affinity level 3 a
shift of 32 would result in shifting out of the 32-bit type and
have no effect on the MPIDR.

These values need to be extended to unsigned long before shifting
to ensure correct results for affinity level 3.

Change-Id: I1ef40afea535f14cfd820c347a065a228e8f4536

All coding style violations have been fixed in a previous patch and
since then, each individual patch has been checked in this regard.
However, the latest version of the checkpatch.pl script from the Linux
kernel is more advanced and it is able to flag new errors in the
Trusted Firmware codebase. This patch fixes them.

Change-Id: I1f332f2440984be85d36b231bb83260368987077

This patch removes the plat_get_max_afflvl() platform API
and instead replaces it with a platform macro PLATFORM_MAX_AFFLVL.
This is done because the maximum affinity level for a platform
is a static value and it is more efficient for it to be defined
as a platform macro.

NOTE: PLATFORM PORTS NEED TO BE UPDATED ON MERGE OF THIS COMMIT

Fixes ARM-Software/tf-issues#265

Change-Id: I31d89b30c2ccda30d28271154d869060d50df7bf

This patch implements the PSCI_FEATURES function which is a mandatory
API in the PSCI 1.0 specification. A capability variable is
constructed during initialization by examining the plat_pm_ops and
spd_pm_ops exported by the platform and the Secure Payload Dispatcher.
This is used by the PSCI FEATURES function to determine which
PSCI APIs are supported by the platform.

Change-Id: I147ffc1bd5d90b469bd3cc4bbe0a20e95c247df7

This patch reworks the PSCI MIGRATE, MIGRATE_INFO_TYPE and
MIGRATE_INFO_UP_CPU support for Trusted Firmware. The
implementation does the appropriate validation of parameters
and invokes the appropriate hook exported by the SPD.

The TSP is a MP Trusted OS. Hence the ability to actually
migrate a Trusted OS has not been implemented. The
corresponding function is not populated in the spd_pm_hooks
structure for the TSPD.

The `spd_pm_ops_t` has undergone changes with this patch.
SPD PORTS MAY NEED TO BE UPDATED.

Fixes ARM-software/tf-issues#249

Change-Id: Iabd87521bf7c530a5e4506b6d3bfd4f1bf87604f

This patch allows the platform to validate the power_state and
entrypoint information from the normal world early on in PSCI
calls so that we can return the error safely. New optional
pm_ops hooks `validate_power_state` and `validate_ns_entrypoint`
are introduced to do this.

As a result of these changes, all the other pm_ops handlers except
the PSCI_ON handler are expected to be successful. Also, the PSCI
implementation will now assert if a PSCI API is invoked without the
corresponding pm_ops handler being registered by the platform.

NOTE : PLATFORM PORTS WILL BREAK ON MERGE OF THIS COMMIT. The
pm hooks have 2 additional optional callbacks and the return type
of the other hooks have changed.

Fixes ARM-Software/tf-issues#229

Change-Id: I036bc0cff2349187c7b8b687b9ee0620aa7e24dc

This patch replaces the internal psci_save_ns_entry() API with a
psci_get_ns_ep_info() API. The new function splits the work done by the
previous one such that it populates and returns an 'entry_point_info_t'
structure with the information to enter the normal world upon completion
of the CPU_SUSPEND or CPU_ON call. This information is used to populate
the non-secure context structure separately.

This allows the new internal API `psci_get_ns_ep_info` to return error
and enable the code to return safely.

Change-Id: Ifd87430a4a3168eac0ebac712f59c93cbad1b231

This patch extends the build option `USE_COHERENT_MEMORY` to
conditionally remove coherent memory from the memory maps of
all boot loader stages. The patch also adds necessary
documentation for coherent memory removal in firmware-design,
porting and user guides.

Fixes ARM-Software/tf-issues#106

Change-Id: I260e8768c6a5c2efc402f5804a80657d8ce38773

This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.

As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.

Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0

This patch fixes the assertion failure when CPU_SUSPEND is invoked with
an affinity level higher than supported by the platform by adding suitable
checks for affinity level within `psci_cpu_suspend`. Also added suitable
bound checks within `psci_aff_map_get_idx` to prevent indexing beyond array
limits.

Fixes ARM-software/tf-issues#260

Change-Id: I04b75c49729e6c6d1983add590f60146c8fc3630

This patch implements the following cleanups in PSCI generic code:

1. It reworks the affinity level specific handlers in the PSCI implementation
   such that.

   a. Usage of the 'rc' local variable is restricted to only where it is
      absolutely needed

   b. 'plat_state' local variable is defined only when a direct invocation of
      plat_get_phys_state() does not suffice.

   c. If a platform handler is not registered then the level specific handler
      returns early.

2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
   arrays of the type instead of using it in function prototypes as well.

3. It removes dangling declarations of __psci_cpu_off() and
   __psci_cpu_suspend(). The definitions of these functions were removed in
   earlier patches.

Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

This patch pulls out state management from the affinity level specific handlers
into the top level functions specific to the operation
i.e. psci_afflvl_suspend(), psci_afflvl_on() etc.

In the power down path this patch will allow an affinity instance at level X to
determine the state that an affinity instance at level X+1 will enter before the
level specific handlers are called. This will be useful to determine whether a
CPU is the last in the cluster during a suspend/off request and so on.

Similarly, in the power up path this patch will allow an affinity instance at
level X to determine the state that an affinity instance at level X+1 has
emerged from, even after the level specific handlers have been called. This will
be useful in determining whether a CPU is the first in the cluster during a
on/resume request and so on.

As before, while powering down, state is updated before the level specific
handlers are invoked so that they can perform actions based upon their target
state. While powering up, state is updated after the level specific handlers have
been invoked so that they can perform actions based upon the state they emerged
from.

Change-Id: I40fe64cb61bb096c66f88f6d493a1931243cfd37

This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.

The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.

The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.

The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.

Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf

This patch adds support for SYSTEM_OFF and SYSTEM_RESET PSCI
operations. A platform should export handlers to complete the
requested operation. The FVP port exports fvp_system_off() and
fvp_system_reset() as an example.

If the SPD provides a power management hook for system off and
system reset, then the SPD is notified about the corresponding
operation so it can do some bookkeeping. The TSPD exports
tspd_system_off() and tspd_system_reset() for that purpose.

Versatile Express shutdown and reset methods have been removed
from the FDT as new PSCI sys_poweroff and sys_reset services
have been added. For those kernels that do not support yet these
PSCI services (i.e. GICv3 kernel), the original dtsi files have
been renamed to *-no_psci.dtsi.

Fixes ARM-software/tf-issues#218

Change-Id: Ic8a3bf801db979099ab7029162af041c4e8330c8

This patch removes the allocation of memory for coherent stacks, associated
accessor function and some dead code which called the accessor function. It also
updates the porting guide to remove the concept and the motivation behind using
stacks allocated in coherent memory.

Fixes ARM-software/tf-issues#198

Change-Id: I00ff9a04f693a03df3627ba39727e3497263fc38

Many of the interfaces internal to PSCI pass the current CPU
MPIDR_EL1 value from function to function. This is not required,
and with inline access to the system registers is less efficient
than requiring the code to read that register whenever required.

This patch remove the mpidr parameter from the affected interfaces
and reduces code in FVP BL3-1 size by 160 bytes.

Change-Id: I16120a7c6944de37232016d7e109976540775602

Exclude stdlib files because they do not follow kernel code style.

Fixes ARM-software/tf-issues#73

Change-Id: I4cfafa38ab436f5ab22c277cb38f884346a267ab

The bakery lock code currently expects the calling code to pass
the MPIDR_EL1 of the current CPU.

This is not always done correctly. Also the change to provide
inline access to system registers makes it more efficient for the
bakery lock code to obtain the MPIDR_EL1 directly.

This change removes the mpidr parameter from the bakery lock
interface, and results in a code reduction of 160 bytes for the
ARM FVP port.

Fixes ARM-software/tf-issues#213

Change-Id: I7ec7bd117bcc9794a0d948990fcf3336a367d543

psci_suspend_context is an array of cache-line aligned structures
containing the single power_state integer per cpu. This array is
the only structure indexed by the aff_map_node.data integer.

This patch saves 2KB of BL3-1 memory by placing the CPU
power_state value directly in the aff_map_node structure. As a
result, this value is now never cached and the cache clean when
writing the value is no longer required.

Fixes ARM-software/tf-issues#195

Change-Id: Ib4c70c8f79eed295ea541e7827977a588a19ef9b

Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
*  The first uses entry_point_info to initialize the relevant
   cpu_context for first entry into a lower exception level on a CPU
*  The second populates the EL1 and EL2 system registers as needed
   from the cpu_context to ensure correct entry into the lower EL

This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).

In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.

Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f

All callers of cm_get_context() pass the calling CPU MPIDR to the
function. Providing a specialised version for the current
CPU results in a reduction in code size and better readability.

The current function has been renamed to cm_get_context_by_mpidr()
and the existing name is now used for the current-CPU version.

The same treatment has been done to cm_set_context(), although
only both forms are used at present in the PSCI and TSPD code.

Change-Id: I91cb0c2f7bfcb950a045dbd9ff7595751c0c0ffb

Previously, platform.h contained many declarations and definitions
used for different purposes. This file has been split so that:

* Platform definitions used by common code that must be defined
  by the platform are now in platform_def.h. The exact include
  path is exported through $PLAT_INCLUDES in the platform makefile.

* Platform definitions specific to the FVP platform are now in
  /plat/fvp/fvp_def.h.

* Platform API declarations specific to the FVP platform are now
  in /plat/fvp/fvp_private.h.

* The remaining platform API declarations that must be ported by
  each platform are still in platform.h but this file has been
  moved to /include/plat/common since this can be shared by all
  platforms.

Change-Id: Ieb3bb22fbab3ee8027413c6b39a783534aee474a

Some data variables were declared but not used. These have been
removed.

Change-Id: I038632af3c32d88984cd25b886c43ff763269bf9

This patch introduces macros (SPSR_64 and SPSR_32) to
create a SPSR for both aarch32 and aarch64 execution
states. These macros allow the user to set fields
in the SPSR depending upon its format.
The make_spsr() function which did not allow
manipulation of all the fields in the aarch32 SPSR
has been replaced by these new macros.

Change-Id: I9425dda0923e8d5f03d03ddb8fa0e28392c4c61e

Update code base to remove variables from the .data section,
mainly by using const static data where possible and adding
the const specifier as required. Most changes are to the IO
subsystem, including the framework APIs. The FVP power
management code is also affected.

Delay initialization of the global static variable,
next_image_type in bl31_main.c, until it is realy needed.
Doing this moves the variable from the .data to the .bss
section.

Also review the IO interface for inconsistencies, using
uintptr_t where possible instead of void *. Remove the
io_handle and io_dev_handle typedefs, which were
unnecessary, replacing instances with uintptr_t.

Fixes ARM-software/tf-issues#107.

Change-Id: I085a62197c82410b566e4698e5590063563ed304

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e

Add tag names to all unnamed structs in header files. This
allows forward declaration of structs, which is necessary to
reduce header file nesting (to be implemented in a subsequent
commit).

Also change the typedef names across the codebase to use the _t
suffix to be more conformant with the Linux coding style. The
coding style actually prefers us not to use typedefs at all but
this is considered a step too far for Trusted Firmware.

Also change the IO framework structs defintions to use typedef'd
structs to be consistent with the rest of the codebase.

Change-Id: I722b2c86fc0d92e4da3b15e5cab20373dd26786f

Make codebase consistent in its use of #include "" syntax for
user includes and #include <> syntax for system includes.

Fixes ARM-software/tf-issues#65

Change-Id: If2f7c4885173b1fd05ac2cde5f1c8a07000c7a33

This patch saves the 'power_state' parameter prior to suspending
a cpu and invalidates it upon its resumption. The 'affinity level'
and 'state id' fields of this parameter can be read using a set of
public and private apis. Validation of power state parameter is
introduced which checks for SBZ bits are zero.
This change also takes care of flushing the parameter from the cache
to main memory. This ensures that it is available after cpu reset
when the caches and mmu are turned off. The earlier support for
saving only the 'affinity level' field of the 'power_state' parameter
has also been reworked.

Fixes ARM-Software/tf-issues#26
Fixes ARM-Software/tf-issues#130

Change-Id: Ic007ccb5e39bf01e0b67390565d3b4be33f5960a

This patch implements ARM Standard Service as a runtime service and adds
support for call count, UID and revision information SMCs. The existing
PSCI implementation is subsumed by the Standard Service calls and all
PSCI calls are therefore dispatched by the Standard Service to the PSCI
handler.

At present, PSCI is the only specification under Standard Service. Thus
call count returns the number of PSCI calls implemented. As this is the
initial implementation, a revision number of 0.1 is returned for call
revision.

Fixes ARM-software/tf-issues#62

Change-Id: I6d4273f72ad6502636efa0f872e288b191a64bc1

At present SPD power management hooks and BL3-2 entry are implemented
using weak references. This would have the handlers bound and registered
with the core framework at build time, but leaves them dangling if a
service fails to initialize at runtime.

This patch replaces implementation by requiring runtime handlers to
register power management and deferred initialization hooks with the
core framework at runtime. The runtime services are to register the
hooks only as the last step, after having all states successfully
initialized.

Change-Id: Ibe788a2a381ef39aec1d4af5ba02376e67269782

This patch implements a set of handlers in the SPD which are called by
the PSCI runtime service upon receiving a power management
operation. These handlers in turn pass control to the Secure Payload
image if required before returning control to PSCI. This ensures that
the Secure Payload has complete visibility of all power transitions in
the system and can prepare accordingly.

Change-Id: I2d1dba5629b7cf2d53999d39fe807dfcf3f62fe2

This patch creates a 'services' directory and moves the PSCI under
it. Other runtime services e.g. the Secure Payload Dispatcher service
will be placed under the same directory in the future.

Also fixes issue ARM-software/tf-issues#12

Change-Id: I187f83dcb660b728f82155d91882e961d2255068

This patch uses the reworked exception handling support to handle
runtime service requests through SMCs following the SMC calling
convention. This is a giant commit since all the changes are
inter-related. It does the following:

1. Replace the old exception handling mechanism with the new one
2. Enforce that SP_EL0 is used C runtime stacks.
3. Ensures that the cold and warm boot paths use the 'cpu_context'
   structure to program an ERET into the next lower EL.
4. Ensures that SP_EL3 always points to the next 'cpu_context'
   structure prior to an ERET into the next lower EL
5. Introduces a PSCI SMC handler which completes the use of PSCI as a
   runtime service

Change-Id: I661797f834c0803d2c674d20f504df1b04c2b852
Co-authored-by: Achin Gupta <achin.gupta@arm.com>

Also fix warnings generated in release builds when assert code
is absent.

Change-Id: I45b9173d3888f9e93e98eb5b4fdc06727ba5cbf4

The psci implementation does not track target affinity level requests
specified during cpu_suspend calls correctly as per the following
example.

1. cpu0.cluster0 calls cpu_suspend with the target affinity level as 0
2. Only the cpu0.cluster0 is powered down while cluster0 remains
   powered up
3. cpu1.cluster0 calls cpu_off to power itself down to highest
   possible affinity level
4. cluster0 will be powered off even though cpu0.cluster0 does not
   allow cluster shutdown

This patch introduces reference counts at affinity levels > 0 to track
the number of cpus which want an affinity instance at level X to
remain powered up. This instance can be turned off only if its
reference count is 0. Cpus still undergo the normal state transitions
(ON, OFF, ON_PENDING, SUSPEND) but the higher levels can only be
either ON or OFF depending upon their reference count.

The above issue is thus fixed as follows:

1. cluster0's reference count is incremented by two when cpu0 and cpu1
   are initially powered on.

2. cpu0.cluster0 calls cpu_suspend with the target affinity level as
   0. This does not affect the cluster0 reference count.

3. Only the cpu0.cluster0 is powered down while cluster0 remains
   powered up as it has a non-zero reference count.

4. cpu1.cluster0 call cpu_off to power itself down to highest possible
   affinity level. This decrements the cluster0 reference count.

5. cluster0 is still not powered off since its reference count will at
   least be 1 due to the restriction placed by cpu0.

Change-Id: I433dfe82b946f5f6985b1602c2de87800504f7a9