GitLab

This patch adds a new optional platform hook `pwr_domain_pwr_down_wfi()` in
the plat_psci_ops structure. This hook allows the platform to perform platform
specific actions including the wfi invocation to enter powerdown. This hook
is invoked by both psci_do_cpu_off() and psci_cpu_suspend_start() functions.
The porting-guide.md is also updated for the same.

This patch also modifies the `psci_power_down_wfi()` function to invoke
`plat_panic_handler` incase of panic instead of the busy while loop.

Fixes ARM-Software/tf-issues#375

Change-Id: Iba104469a1445ee8d59fb3a6fdd0a98e7f24dfa3

Change-Id: I6f49bd779f2a4d577c6443dd160290656cdbc59b

This patch unifies the bakery lock api's across coherent and normal
memory implementation of locks by using same data type `bakery_lock_t`
and similar arguments to functions.

A separate section `bakery_lock` has been created and used to allocate
memory for bakery locks using `DEFINE_BAKERY_LOCK`. When locks are
allocated in normal memory, each lock for a core has to spread
across multiple cache lines. By using the total size allocated in a
separate cache line for a single core at compile time, the memory for
other core locks is allocated at link time by multiplying the single
core locks size with (PLATFORM_CORE_COUNT - 1). The normal memory lock
algorithm now uses lock address instead of the `id` in the per_cpu_data.
For locks allocated in coherent memory, it moves locks from
tzfw_coherent_memory to bakery_lock section.

The bakery locks are allocated as part of bss or in coherent memory
depending on usage of coherent memory. Both these regions are
initialised to zero as part of run_time_init before locks are used.
Hence, bakery_lock_init() is made an empty function as the lock memory
is already initialised to zero.

The above design lead to the removal of psci bakery locks from
non_cpu_power_pd_node to psci_locks.

NOTE: THE BAKERY LOCK API WHEN USE_COHERENT_MEM IS NOT SET HAS CHANGED.
THIS IS A BREAKING CHANGE FOR ALL PLATFORM PORTS THAT ALLOCATE BAKERY
LOCKS IN NORMAL MEMORY.

Change-Id: Ic3751c0066b8032dcbf9d88f1d4dc73d15f61d8b

In certain Trusted OS implementations it is a requirement to pass them the
highest power level which will enter a power down state during a PSCI
CPU_SUSPEND or SYSTEM_SUSPEND API invocation. This patch passes this power level
to the SPD in the "max_off_pwrlvl" parameter of the svc_suspend() hook.

Currently, the highest power level which was requested to be placed in a low
power state (retention or power down) is passed to the SPD svc_suspend_finish()
hook. This hook is called after emerging from the low power state. It is more
useful to pass the highest power level which was powered down instead. This
patch does this by changing the semantics of the parameter passed to an SPD's
svc_suspend_finish() hook. The name of the parameter has been changed from
"suspend_level" to "max_off_pwrlvl" as well. Same changes have been made to the
parameter passed to the tsp_cpu_resume_main() function.

NOTE: THIS PATCH CHANGES THE SEMANTICS OF THE EXISTING "svc_suspend_finish()"
      API BETWEEN THE PSCI AND SPD/SP IMPLEMENTATIONS. THE LATTER MIGHT NEED
      UPDATES TO ENSURE CORRECT BEHAVIOUR.

Change-Id: If3a9d39b13119bbb6281f508a91f78a2f46a8b90

This patch reworks the PSCI generic implementation to conform to ARM
Trusted Firmware coding guidelines as described here:
https://github.com/ARM-software/arm-trusted-firmware/wiki

This patch also reviews the use of signed data types within PSCI
Generic code and replaces them with their unsigned counterparts wherever
they are not appropriate. The PSCI_INVALID_DATA macro which was defined
to -1 is now replaced with PSCI_INVALID_PWR_LVL macro which is defined
to PLAT_MAX_PWR_LVL + 1.

Change-Id: Iaea422d0e46fc314e0b173c2b4c16e0d56b2515a

As per PSCI1.0 specification, the error code to be returned when an invalid
non secure entrypoint address is specified by the PSCI client for CPU_SUSPEND,
CPU_ON or SYSTEM_SUSPEND must be PSCI_E_INVALID_ADDRESS. The current PSCI
implementation returned PSCI_E_INVAL_PARAMS. This patch rectifies this error
and also implements a common helper function to validate the entrypoint
information to be used across these PSCI API implementations.

Change-Id: I52d697d236c8bf0cd3297da4008c8e8c2399b170

This commit does the switch to the new PSCI framework implementation replacing
the existing files in PSCI folder with the ones in PSCI1.0 folder. The
corresponding makefiles are modified as required for the new implementation.
The platform.h header file is also is switched to the new one
as required by the new frameworks. The build flag ENABLE_PLAT_COMPAT defaults
to 1 to enable compatibility layer which let the existing platform ports to
continue to build and run with minimal changes.

The default weak implementation of platform_get_core_pos() is now removed from
platform_helpers.S and is provided by the compatibility layer.

Note: The Secure Payloads and their dispatchers still use the old platform
and framework APIs and hence it is expected that the ENABLE_PLAT_COMPAT build
flag will remain enabled in subsequent patch. The compatibility for SPDs using
the older APIs on platforms migrated to the new APIs will be added in the
following patch.

Change-Id: I18c51b3a085b564aa05fdd98d11c9f3335712719

The new PSCI topology framework and PSCI extended State framework introduces
a breaking change in the platform port APIs. To ease the migration of the
platform ports to the new porting interface, a compatibility layer is
introduced which essentially defines the new platform API in terms of the
old API. The old PSCI helpers to retrieve the power-state, its associated
fields and the highest coordinated physical OFF affinity level of a core
are also implemented for compatibility. This allows the existing
platform ports to work with the new PSCI framework without significant
rework. This layer will be enabled by default once the switch to the new
PSCI framework is done and is controlled by the build flag ENABLE_PLAT_COMPAT.

Change-Id: I4b17cac3a4f3375910a36dba6b03d8f1700d07e3

There used to be 2 warm reset entry points:

 - the "on finisher", for when the core has been turned on using a
   PSCI CPU_ON call;

 - the "suspend finisher", entered upon resumption from a previous
   PSCI CPU_SUSPEND call.

The appropriate warm reset entry point used to be programmed into the
mailboxes by the power management hooks.

However, it is not required to provide this information to the PSCI
entry point code, as it can figure it out by itself. By querying affinity
info state, a core is able to determine on which execution path it is.
If the state is ON_PENDING then it means it's been turned on else
it is resuming from suspend.

This patch unifies the 2 warm reset entry points into a single one:
psci_entrypoint(). The patch also implements the necessary logic
to distinguish between the 2 types of warm resets in the power up
finisher.

The plat_setup_psci_ops() API now takes the
secure entry point as an additional parameter to enable the platforms
to configure their mailbox. The platform hooks `pwr_domain_on`
and `pwr_domain_suspend` no longer take secure entry point as
a parameter.

Change-Id: I7d1c93787b54213aefdbc046b8cd66a555dfbfd9

The state-id field in the power-state parameter of a CPU_SUSPEND call can be
used to describe composite power states specific to a platform. The current PSCI
implementation does not interpret the state-id field. It relies on the target
power level and the state type fields in the power-state parameter to perform
state coordination and power management operations. The framework introduced
in this patch allows the PSCI implementation to intepret generic global states
like RUN, RETENTION or OFF from the State-ID to make global state coordination
decisions and reduce the complexity of platform ports. It adds support to
involve the platform in state coordination which facilitates the use of
composite power states and improves the support for entering standby states
at multiple power domains.

The patch also includes support for extended state-id format for the power
state parameter as specified by PSCIv1.0.

The PSCI implementation now defines a generic representation of the power-state
parameter. It depends on the platform port to convert the power-state parameter
(possibly encoding a composite power state) passed in a CPU_SUSPEND call to this
representation via the `validate_power_state()` plat_psci_ops handler. It is an
array where each index corresponds to a power level. Each entry contains the
local power state the power domain at that power level could enter.

The meaning of the local power state values is platform defined, and may vary
between levels in a single platform. The PSCI implementation constrains the
values only so that it can classify the state as RUN, RETENTION or OFF as
required by the specification:
   * zero means RUN
   * all OFF state values at all levels must be higher than all RETENTION
     state values at all levels
   * the platform provides PLAT_MAX_RET_STATE and PLAT_MAX_OFF_STATE values
     to the framework

The platform also must define the macros PLAT_MAX_RET_STATE and
PLAT_MAX_OFF_STATE which lets the PSCI implementation find out which power
domains have been requested to enter a retention or power down state. The PSCI
implementation does not interpret the local power states defined by the
platform. The only constraint is that the PLAT_MAX_RET_STATE <
PLAT_MAX_OFF_STATE.

For a power domain tree, the generic implementation maintains an array of local
power states. These are the states requested for each power domain by all the
cores contained within the domain. During a request to place multiple power
domains in a low power state, the platform is passed an array of requested
power-states for each power domain through the plat_get_target_pwr_state()
API. It coordinates amongst these states to determine a target local power
state for the power domain. A default weak implementation of this API is
provided in the platform layer which returns the minimum of the requested
power-states back to the PSCI state coordination.

Finally, the plat_psci_ops power management handlers are passed the target
local power states for each affected power domain using the generic
representation described above. The platform executes operations specific to
these target states.

The platform power management handler for placing a power domain in a standby
state (plat_pm_ops_t.pwr_domain_standby()) is now only used as a fast path for
placing a core power domain into a standby or retention state should now be
used to only place the core power domain in a standby or retention state.

The extended state-id power state format can be enabled by setting the
build flag PSCI_EXTENDED_STATE_ID=1 and it is disabled by default.

Change-Id: I9d4123d97e179529802c1f589baaa4101759d80c

This patch removes the assumption in the current PSCI implementation that MPIDR
based affinity levels map directly to levels in a power domain tree. This
enables PSCI generic code to support complex power domain topologies as
envisaged by PSCIv1.0 specification. The platform interface for querying
the power domain topology has been changed such that:

1. The generic PSCI code does not generate MPIDRs and use them to query the
   platform about the number of power domains at a particular power level. The
   platform now provides a description of the power domain tree on the SoC
   through a data structure. The existing platform APIs to provide the same
   information have been removed.

2. The linear indices returned by plat_core_pos_by_mpidr() and
   plat_my_core_pos() are used to retrieve core power domain nodes from the
   power domain tree. Power domains above the core level are accessed using a
   'parent' field in the tree node descriptors.

The platform describes the power domain tree in an array of 'unsigned
char's. The first entry in the array specifies the number of power domains at
the highest power level implemented in the system. Each susbsequent entry
corresponds to a power domain and contains the number of power domains that are
its direct children. This array is exported to the generic PSCI implementation
via the new `plat_get_power_domain_tree_desc()` platform API.

The PSCI generic code uses this array to populate its internal power domain tree
using the Breadth First Search like algorithm. The tree is split into two
arrays:

1. An array that contains all the core power domain nodes

2. An array that contains all the other power domain nodes

A separate array for core nodes allows certain core specific optimisations to
be implemented e.g. remove the bakery lock, re-use per-cpu data framework for
storing some information.

Entries in the core power domain array are allocated such that the
array index of the domain is equal to the linear index returned by
plat_core_pos_by_mpidr() and plat_my_core_pos() for the MPIDR
corresponding to that domain. This relationship is key to be able to use
an MPIDR to find the corresponding core power domain node, traverse to higher
power domain nodes and index into arrays that contain core specific
information.

An introductory document has been added to briefly describe the new interface.

Change-Id: I4b444719e8e927ba391cae48a23558308447da13

As per Section 4.2.2. in the PSCI specification, the term "affinity"
is used in the context of describing the hierarchical arrangement
of cores. This often, but not always, maps directly to the processor
power domain topology of the system. The current PSCI implementation
assumes that this is always the case i.e. MPIDR based levels of
affinity always map to levels in a power domain topology tree.

This patch is the first in a series of patches which remove this
assumption. It removes all occurences of the terms "affinity
instances and levels" when used to describe the power domain
topology. Only the terminology is changed in this patch. Subsequent
patches will implement functional changes to remove the above
mentioned assumption.

Change-Id: Iee162f051b228828310610c5a320ff9d31009b4e

This patch optimizes the invocation of the platform power management hooks for
ON, OFF and SUSPEND such that they are called only for the highest affinity
level which will be powered off/on. Earlier, the hooks were being invoked for
all the intermediate levels as well.

This patch requires that the platforms migrate to the new semantics of the PM
hooks.  It also removes the `state` parameter from the pm hooks as the `afflvl`
parameter now indicates the highest affinity level for which power management
operations are required.

Change-Id: I57c87931d8a2723aeade14acc710e5b78ac41732

This patch creates a copy of the existing PSCI files and related psci.h and
platform.h header files in a new `PSCI1.0` directory. The changes for the
new PSCI power domain topology and extended state-ID frameworks will be
added incrementally to these files. This incremental approach will
aid in review and in understanding the changes better. Once all the
changes have been introduced, these files will replace the existing PSCI
files.

Change-Id: Ibb8a52e265daa4204e34829ed050bddd7e3316ff

This patch adds support for SYSTEM_SUSPEND API as mentioned in the PSCI 1.0
specification. This API, on being invoked on the last running core on a
supported platform, will put the system into a low power mode with memory
retention.

The psci_afflvl_suspend() internal API has been reused as most of the actions
to suspend a system are the same as invoking the PSCI CPU_SUSPEND API with the
target affinity level as 'system'. This API needs the 'power state' parameter
for the target low power state. This parameter is not passed by the caller of
the SYSTEM_SUSPEND API. Hence, the platform needs to implement the
get_sys_suspend_power_state() platform function to provide this information.
Also, the platform also needs to add support for suspending the system to the
existing 'plat_pm_ops' functions: affinst_suspend() and
affinst_suspend_finish().

Change-Id: Ib6bf10809cb4e9b92f463755608889aedd83cef5

This patch:

   * Bumps the PSCI VERSION to 1.0. This means that
     the PSCI_VERSION API will now return the value 0x00010000
     to indicate the version as 1.0. The firmware remains
     compatible with PSCI v0.2 clients.

   * The firmware design guide is updated to document the
     APIs supported by the Trusted Firmware generic code.

   * The FVP Device Tree Sources (dts) and Blobs(dtb) are also
     updated to add "psci-1.0" and "psci-0.2" to the list of
     compatible PSCI versions.

Change-Id: Iafc2f549c92651dcd65d7e24a8aae35790d00f8a

This patch implements the PSCI_FEATURES function which is a mandatory
API in the PSCI 1.0 specification. A capability variable is
constructed during initialization by examining the plat_pm_ops and
spd_pm_ops exported by the platform and the Secure Payload Dispatcher.
This is used by the PSCI FEATURES function to determine which
PSCI APIs are supported by the platform.

Change-Id: I147ffc1bd5d90b469bd3cc4bbe0a20e95c247df7

This patch reworks the PSCI MIGRATE, MIGRATE_INFO_TYPE and
MIGRATE_INFO_UP_CPU support for Trusted Firmware. The
implementation does the appropriate validation of parameters
and invokes the appropriate hook exported by the SPD.

The TSP is a MP Trusted OS. Hence the ability to actually
migrate a Trusted OS has not been implemented. The
corresponding function is not populated in the spd_pm_hooks
structure for the TSPD.

The `spd_pm_ops_t` has undergone changes with this patch.
SPD PORTS MAY NEED TO BE UPDATED.

Fixes ARM-software/tf-issues#249

Change-Id: Iabd87521bf7c530a5e4506b6d3bfd4f1bf87604f

This patch allows the platform to validate the power_state and
entrypoint information from the normal world early on in PSCI
calls so that we can return the error safely. New optional
pm_ops hooks `validate_power_state` and `validate_ns_entrypoint`
are introduced to do this.

As a result of these changes, all the other pm_ops handlers except
the PSCI_ON handler are expected to be successful. Also, the PSCI
implementation will now assert if a PSCI API is invoked without the
corresponding pm_ops handler being registered by the platform.

NOTE : PLATFORM PORTS WILL BREAK ON MERGE OF THIS COMMIT. The
pm hooks have 2 additional optional callbacks and the return type
of the other hooks have changed.

Fixes ARM-Software/tf-issues#229

Change-Id: I036bc0cff2349187c7b8b687b9ee0620aa7e24dc

This patch adds support to save the "power state" parameter before the
affinity level specific handlers are called in a CPU_SUSPEND call.
This avoids the need to pass the power_state as a parameter to the
handlers and Secure Payload Dispatcher (SPD) suspend spd_pm_ops.
The power_state arguments in the spd_pm_ops operations are now reserved
and must not be used. The SPD can query the relevant power_state fields
by using the psci_get_suspend_afflvl() & psci_get_suspend_stateid() APIs.

NOTE: THIS PATCH WILL BREAK THE SPD_PM_OPS INTERFACE. HENCE THE SECURE
PAYLOAD DISPATCHERS WILL NEED TO BE REWORKED TO USE THE NEW INTERFACE.

Change-Id: I1293d7dc8cf29cfa6a086a009eee41bcbf2f238e

This patch removes the non-secure entry point information being passed
to the platform pm_ops which is not needed. Also, it removes the `mpidr`
parameter for  platform pm hooks which are meant to do power management
operations only on the current cpu.

NOTE: PLATFORM PORTS MUST BE UPDATED AFTER MERGING THIS COMMIT.

Change-Id: If632376a990b7f3b355f910e78771884bf6b12e7

This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.

As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.

Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0

This patch implements the following cleanups in PSCI generic code:

1. It reworks the affinity level specific handlers in the PSCI implementation
   such that.

   a. Usage of the 'rc' local variable is restricted to only where it is
      absolutely needed

   b. 'plat_state' local variable is defined only when a direct invocation of
      plat_get_phys_state() does not suffice.

   c. If a platform handler is not registered then the level specific handler
      returns early.

2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
   arrays of the type instead of using it in function prototypes as well.

3. It removes dangling declarations of __psci_cpu_off() and
   __psci_cpu_suspend(). The definitions of these functions were removed in
   earlier patches.

Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.

The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.

The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.

The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.

Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf

This patch adds support for SYSTEM_OFF and SYSTEM_RESET PSCI
operations. A platform should export handlers to complete the
requested operation. The FVP port exports fvp_system_off() and
fvp_system_reset() as an example.

If the SPD provides a power management hook for system off and
system reset, then the SPD is notified about the corresponding
operation so it can do some bookkeeping. The TSPD exports
tspd_system_off() and tspd_system_reset() for that purpose.

Versatile Express shutdown and reset methods have been removed
from the FDT as new PSCI sys_poweroff and sys_reset services
have been added. For those kernels that do not support yet these
PSCI services (i.e. GICv3 kernel), the original dtsi files have
been renamed to *-no_psci.dtsi.

Fixes ARM-software/tf-issues#218

Change-Id: Ic8a3bf801db979099ab7029162af041c4e8330c8

The array of affinity nodes is currently allocated for 32 entries
with the PSCI_NUM_AFFS value defined in psci.h. This is not enough
for large systems, and will substantially over allocate the array
for small systems.

This patch introduces an optional platform definition
PLATFORM_NUM_AFFS to platform_def.h. If defined this value is
used for PSCI_NUM_AFFS, otherwise a value of two times the number
of CPU cores is used.

The FVP port defines PLATFORM_NUM_AFFS to be 10 which saves
nearly 1.5KB of memory.

Fixes ARM-software/tf-issues#192

Change-Id: I68e30ac950de88cfbd02982ba882a18fb69c1445

Function declarations implicitly have external linkage so do not
need the extern keyword.

Change-Id: Ia0549786796d8bf5956487e8996450a0b3d79f32

This patch fixes the broken support for entry into standby states
introduced under commit-id 'd118f9f8' (tf-issues#94). Upon exit from
the platform defined standby state instead of returning to the caller
of the SMC, execution would get stuck in the wfi instruction meant for
entering a power down state. This patch ensures that exit from a
standby state and entry into a power down state do not interfere with
each other.

Fixes ARM-software/tf-issues#154

Change-Id: I56e5df353368e44d6eefc94ffedefe21929f5cfe

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e

Add tag names to all unnamed structs in header files. This
allows forward declaration of structs, which is necessary to
reduce header file nesting (to be implemented in a subsequent
commit).

Also change the typedef names across the codebase to use the _t
suffix to be more conformant with the Linux coding style. The
coding style actually prefers us not to use typedefs at all but
this is considered a step too far for Trusted Firmware.

Also change the IO framework structs defintions to use typedef'd
structs to be consistent with the rest of the codebase.

Change-Id: I722b2c86fc0d92e4da3b15e5cab20373dd26786f

Move the PSCI global functions out of psci_private.h and into
psci.h to allow the standard service to only depend on psci.h.

Change-Id: I8306924a3814b46e70c1dcc12524c7aefe06eed1

Move almost all system include files to a logical sub-directory
under ./include. The only remaining system include directories
not under ./include are specific to the platform. Move the
corresponding source files to match the include directory
structure.

Also remove pm.h as it is no longer used.

Change-Id: Ie5ea6368ec5fad459f3e8a802ad129135527f0b3