GitLab

The cpu-ops pointer was initialized before enabling the data cache in the cold
and warm boot paths. This required a DCIVAC cache maintenance operation to
invalidate any stale cache lines resident in other cpus.

This patch moves this initialization to the bl31_arch_setup() function
which is always called after the data cache and MMU has been enabled.

This change removes the need:
 1. for the DCIVAC cache maintenance operation.
 2. to initialise the CPU ops upon resumption from a PSCI CPU_SUSPEND
    call since memory contents are always preserved in this case.

Change-Id: Ibb2fa2f7460d1a1f1e721242025e382734c204c6

This patch provides an option to specify a interrupt routing model
where non-secure interrupts (IRQs) are routed to EL3 instead of S-EL1.
When such an interrupt occurs, the TSPD arranges a return to
the normal world after saving any necessary context. The interrupt
routing model to route IRQs to EL3 is enabled only during STD SMC
processing. Thus the pre-emption of S-EL1 is disabled during Fast SMC
and Secure Interrupt processing.

A new build option TSPD_ROUTE_NS_INT_EL3 is introduced to change
the non secure interrupt target execution level to EL3.

Fixes ARM-software/tf-issues#225

Change-Id: Ia1e779fbbb6d627091e665c73fa6315637cfdd32

This patch:

   * Bumps the PSCI VERSION to 1.0. This means that
     the PSCI_VERSION API will now return the value 0x00010000
     to indicate the version as 1.0. The firmware remains
     compatible with PSCI v0.2 clients.

   * The firmware design guide is updated to document the
     APIs supported by the Trusted Firmware generic code.

   * The FVP Device Tree Sources (dts) and Blobs(dtb) are also
     updated to add "psci-1.0" and "psci-0.2" to the list of
     compatible PSCI versions.

Change-Id: Iafc2f549c92651dcd65d7e24a8aae35790d00f8a

This patch implements the PSCI_FEATURES function which is a mandatory
API in the PSCI 1.0 specification. A capability variable is
constructed during initialization by examining the plat_pm_ops and
spd_pm_ops exported by the platform and the Secure Payload Dispatcher.
This is used by the PSCI FEATURES function to determine which
PSCI APIs are supported by the platform.

Change-Id: I147ffc1bd5d90b469bd3cc4bbe0a20e95c247df7

This patch reworks the PSCI MIGRATE, MIGRATE_INFO_TYPE and
MIGRATE_INFO_UP_CPU support for Trusted Firmware. The
implementation does the appropriate validation of parameters
and invokes the appropriate hook exported by the SPD.

The TSP is a MP Trusted OS. Hence the ability to actually
migrate a Trusted OS has not been implemented. The
corresponding function is not populated in the spd_pm_hooks
structure for the TSPD.

The `spd_pm_ops_t` has undergone changes with this patch.
SPD PORTS MAY NEED TO BE UPDATED.

Fixes ARM-software/tf-issues#249

Change-Id: Iabd87521bf7c530a5e4506b6d3bfd4f1bf87604f

This patch allows the platform to validate the power_state and
entrypoint information from the normal world early on in PSCI
calls so that we can return the error safely. New optional
pm_ops hooks `validate_power_state` and `validate_ns_entrypoint`
are introduced to do this.

As a result of these changes, all the other pm_ops handlers except
the PSCI_ON handler are expected to be successful. Also, the PSCI
implementation will now assert if a PSCI API is invoked without the
corresponding pm_ops handler being registered by the platform.

NOTE : PLATFORM PORTS WILL BREAK ON MERGE OF THIS COMMIT. The
pm hooks have 2 additional optional callbacks and the return type
of the other hooks have changed.

Fixes ARM-Software/tf-issues#229

Change-Id: I036bc0cff2349187c7b8b687b9ee0620aa7e24dc

This patch adds support to save the "power state" parameter before the
affinity level specific handlers are called in a CPU_SUSPEND call.
This avoids the need to pass the power_state as a parameter to the
handlers and Secure Payload Dispatcher (SPD) suspend spd_pm_ops.
The power_state arguments in the spd_pm_ops operations are now reserved
and must not be used. The SPD can query the relevant power_state fields
by using the psci_get_suspend_afflvl() & psci_get_suspend_stateid() APIs.

NOTE: THIS PATCH WILL BREAK THE SPD_PM_OPS INTERFACE. HENCE THE SECURE
PAYLOAD DISPATCHERS WILL NEED TO BE REWORKED TO USE THE NEW INTERFACE.

Change-Id: I1293d7dc8cf29cfa6a086a009eee41bcbf2f238e

This patch removes the non-secure entry point information being passed
to the platform pm_ops which is not needed. Also, it removes the `mpidr`
parameter for  platform pm hooks which are meant to do power management
operations only on the current cpu.

NOTE: PLATFORM PORTS MUST BE UPDATED AFTER MERGING THIS COMMIT.

Change-Id: If632376a990b7f3b355f910e78771884bf6b12e7

This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.

As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.

Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0

This patch fixes a crash due to corruption of cpu_ops
data structure. During the secondary CPU boot, after the
cpu_ops has been initialized in the per cpu-data, the
dcache lines need to invalidated so that the update in
memory can be seen later on when the dcaches are turned ON.
Also, after initializing the psci per cpu data, the dcache
lines are flushed so that they are written back to memory
and dirty dcache lines are avoided.

Fixes ARM-Software/tf-issues#271

Change-Id: Ia90f55e9882690ead61226eea5a5a9146d35f313

This patch introduces a framework which will allow CPUs to perform
implementation defined actions after a CPU reset, during a CPU or cluster power
down, and when a crash occurs. CPU specific reset handlers have been implemented
in this patch. Other handlers will be implemented in subsequent patches.

Also moved cpu_helpers.S to the new directory lib/cpus/aarch64/.

Change-Id: I1ca1bade4d101d11a898fb30fea2669f9b37b956

This patch implements the following cleanups in PSCI generic code:

1. It reworks the affinity level specific handlers in the PSCI implementation
   such that.

   a. Usage of the 'rc' local variable is restricted to only where it is
      absolutely needed

   b. 'plat_state' local variable is defined only when a direct invocation of
      plat_get_phys_state() does not suffice.

   c. If a platform handler is not registered then the level specific handler
      returns early.

2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
   arrays of the type instead of using it in function prototypes as well.

3. It removes dangling declarations of __psci_cpu_off() and
   __psci_cpu_suspend(). The definitions of these functions were removed in
   earlier patches.

Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978

This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.

It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.

For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.

This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.

Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6

This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.

The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.

The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.

The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.

Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf

This patch adds a macro which will flush the contents of the specified member of
the per-CPU data structure to the PoC. This is required to enable an update of a
per-CPU data member to be visible to all observers.

Change-Id: I20e0feb9b9f345dc5a1162e88adc7956a7ad7a64

This patch adds support for SYSTEM_OFF and SYSTEM_RESET PSCI
operations. A platform should export handlers to complete the
requested operation. The FVP port exports fvp_system_off() and
fvp_system_reset() as an example.

If the SPD provides a power management hook for system off and
system reset, then the SPD is notified about the corresponding
operation so it can do some bookkeeping. The TSPD exports
tspd_system_off() and tspd_system_reset() for that purpose.

Versatile Express shutdown and reset methods have been removed
from the FDT as new PSCI sys_poweroff and sys_reset services
have been added. For those kernels that do not support yet these
PSCI services (i.e. GICv3 kernel), the original dtsi files have
been renamed to *-no_psci.dtsi.

Fixes ARM-software/tf-issues#218

Change-Id: Ic8a3bf801db979099ab7029162af041c4e8330c8

This patch further optimizes the EL3 register state stored in
cpu_context. The 2 registers which are removed from cpu_context are:

  * cntfrq_el0 is the system timer register which is writable
    only in EL3 and it can be programmed during cold/warm boot. Hence
    it need not be saved to cpu_context.

  * cptr_el3 controls access to Trace, Floating-point, and Advanced
    SIMD functionality and it is programmed every time during cold
    and warm boot. The current BL3-1 implementation does not need to
    modify the access controls during normal execution and hence
    they are expected to remain static.

Fixes ARM-software/tf-issues#197

Change-Id: I599ceee3b73a7dcfd37069fd41b60e3d397a7b18

This patch reworks the crash reporting mechanism to further
optimise the stack and code size. The reporting makes use
of assembly console functions to avoid calling C Runtime
to report the CPU state. The crash buffer requirement is
reduced to 64 bytes with this implementation. The crash
buffer is now part of per-cpu data which makes retrieving
the crash buffer trivial.

Also now panic() will use crash reporting if
invoked from BL3-1.

Fixes ARM-software/tf-issues#199

Change-Id: I79d27a4524583d723483165dc40801f45e627da5

This patch uses stacks allocated in normal memory to enable the MMU early in the
warm boot path thus removing the dependency on stacks allocated in coherent
memory. Necessary cache and stack maintenance is performed when a cpu is being
powered down and up. This avoids any coherency issues that can arise from
reading speculatively fetched stale stack memory from another CPUs cache. These
changes affect the warm boot path in both BL3-1 and BL3-2.

The EL3 system registers responsible for preserving the MMU state are not saved
and restored any longer. Static values are used to program these system
registers when a cpu is powered on or resumed from suspend.

Change-Id: I8357e2eb5eb6c5f448492c5094b82b8927603784

CTX_INCLUDE_FPREGS make variable allows us to include or exclude FP
registers from context structure, in case FP is not used by TSPD.

Fixes ARM-software/tf-issues#194

Change-Id: Iee41af382d691340c7ae21830ad1bbf95dad1f4b

Exclude stdlib files because they do not follow kernel code style.

Fixes ARM-software/tf-issues#73

Change-Id: I4cfafa38ab436f5ab22c277cb38f884346a267ab

The array of affinity nodes is currently allocated for 32 entries
with the PSCI_NUM_AFFS value defined in psci.h. This is not enough
for large systems, and will substantially over allocate the array
for small systems.

This patch introduces an optional platform definition
PLATFORM_NUM_AFFS to platform_def.h. If defined this value is
used for PSCI_NUM_AFFS, otherwise a value of two times the number
of CPU cores is used.

The FVP port defines PLATFORM_NUM_AFFS to be 10 which saves
nearly 1.5KB of memory.

Fixes ARM-software/tf-issues#192

Change-Id: I68e30ac950de88cfbd02982ba882a18fb69c1445

Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
*  The first uses entry_point_info to initialize the relevant
   cpu_context for first entry into a lower exception level on a CPU
*  The second populates the EL1 and EL2 system registers as needed
   from the cpu_context to ensure correct entry into the lower EL

This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).

In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.

Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f

The crash reporting support and early initialisation of the
cpu_data allow the runtime_exception vectors to be used from
the start in BL3-1, removing the need for the additional
early_exception vectors and 2KB of code from BL3-1.

Change-Id: I5f8997dabbaafd8935a7455910b7db174a25d871

Moving the context pointers for each CPU into the per-cpu data
allows for much more efficient access to the contexts for the
current CPU.

Change-Id: Id784e210d63cbdcddb44ac1591617ce668dbc29f

This patch prepares the per-cpu pointer cache for wider use by:
* renaming the structure to cpu_data and placing in new header
* providing accessors for this CPU, or other CPUs
* splitting the initialization of the TPIDR pointer from the
  initialization of the cpu_data content
* moving the crash stack initialization to a crash stack function
* setting the TPIDR pointer very early during boot

Change-Id: Icef9004ff88f8eb241d48c14be3158087d7e49a3

All callers of cm_get_context() pass the calling CPU MPIDR to the
function. Providing a specialised version for the current
CPU results in a reduction in code size and better readability.

The current function has been renamed to cm_get_context_by_mpidr()
and the existing name is now used for the current-CPU version.

The same treatment has been done to cm_set_context(), although
only both forms are used at present in the PSCI and TSPD code.

Change-Id: I91cb0c2f7bfcb950a045dbd9ff7595751c0c0ffb

Rename the ic_* platform porting functions to plat_ic_* to be
consistent with the other functions in platform.h. Also rename
bl31_get_next_image_info() to bl31_plat_get_next_image_ep_info()
and remove the duplicate declaration in bl31.h.

Change-Id: I4851842069d3cff14c0a468daacc0a891a7ede84

Some platform porting functions were in BL specific header files.
These have been moved to platform.h so that all porting functions
are in the same place. The functions are now grouped by BL.
Obsolete BL headers files have been removed.

Also, the weak declaration of the init_bl2_mem_layout() function
has been moved out the header file and into the source file
(bl_common.c) using the more succinct #pragma syntax. This
mitigates the risk of 2 weak definitions being created and the
wrong one being picked up by the compiler.

Change-Id: Ib19934939fd755f3e5a5a5bceec88da684308a83

Some data variables were declared but not used. These have been
removed.

Change-Id: I038632af3c32d88984cd25b886c43ff763269bf9

Function declarations implicitly have external linkage so do not
need the extern keyword.

Change-Id: Ia0549786796d8bf5956487e8996450a0b3d79f32

Implements support for Non Secure Interrupts preempting the
Standard SMC call in EL1. Whenever an IRQ is trapped in the
Secure world we securely handover to the Normal world
to process the interrupt. The normal world then issues
"resume" smc call to resume the previous interrupted SMC call.
Fixes ARM-software/tf-issues#105

Change-Id: I72b760617dee27438754cdfc9fe9bcf4cc024858

This patch adds support in the TSPD for registering a handler for
S-EL1 interrupts. This handler ferries the interrupts generated in the
non-secure state to the TSP at 'tsp_fiq_entry'. Support has been added
to the smc handler to resume execution in the non-secure state once
interrupt handling has been completed by the TSP.

There is also support for resuming execution in the normal world if
the TSP receives a EL3 interrupt. This code is currently unused.

Change-Id: I816732595a2635e299572965179f11aa0bf93b69

This patch adds a common handler for FIQ and IRQ exceptions in the
BL3-1 runtime exception vector table. This function determines the
interrupt type and calls its handler. A crash is reported if an
inconsistency in the interrupt management framework is detected. In
the event of a spurious interrupt, execution resumes from the
instruction where the interrupt was generated.

This patch also removes 'cm_macros.S' as its contents have been moved
to 'runtime_exceptions.S'

Change-Id: I3c85ecf8eaf43a3fac429b119ed0bd706d2e2093

This patch introduces a framework for registering interrupts routed to
EL3. The interrupt routing model is governed by the SCR_EL3.IRQ and
FIQ bits and the security state an interrupt is generated in. The
framework recognizes three type of interrupts depending upon which
exception level and security state they should be handled in
i.e. Secure EL1 interrupts, Non-secure interrupts and EL3
interrupts. It provides an API and macros that allow a runtime service
to register an handler for a type of interrupt and specify the routing
model. The framework validates the routing model and uses the context
management framework to ensure that it is applied to the SCR_EL3 prior
to entry into the target security state. It saves the handler in
internal data structures. An API is provided to retrieve the handler
when an interrupt of a particular type is asserted. Registration is
expected to be done once by the primary CPU. The same handler and
routing model is used for all CPUs.

Support for EL3 interrupts will be added to the framework in the
future. A makefile flag has been added to allow the FVP port choose
between ARM GIC v2 and v3 support in EL3. The latter version is
currently unsupported.

A framework for handling interrupts in BL3-1 will be introduced in
subsequent patches. The default routing model in the absence of any
handlers expects no interrupts to be routed to EL3.

Change-Id: Idf7c023b34fcd4800a5980f2bef85e4b5c29e649

This patch adds an API to write to any bit in the SCR_EL3 member of
the 'cpu_context' structure of the current CPU for a specified
security state. This API will be used in subsequent patches which
introduce interrupt management in EL3 to specify the interrupt routing
model when execution is not in EL3.

It also renames the cm_set_el3_elr() function to cm_set_elr_el3()
which is more in line with the system register name being targeted by
the API.

Change-Id: I310fa7d8f827ad3f350325eca2fb28cb350a85ed

The issues addressed in this patch are:

1. Remove meminfo_t from the common interfaces in BL3-x,
expecting that platform code will find a suitable mechanism
to determine the memory extents in these images and provide
it to the BL3-x images.

2. Remove meminfo_t and bl31_plat_params_t from all FVP BL3-x
code as the images use link-time information to determine
memory extents.

meminfo_t is still used by common interface in BL1/BL2 for
loading images

Change-Id: I4e825ebf6f515b59d84dc2bdddf6edbf15e2d60f

This patch is based on spec published at
https://github.com/ARM-software/tf-issues/issues/133

It rearranges the bl31_args struct into
bl31_params and bl31_plat_params which provide the
information needed for Trusted firmware and platform
specific data via x0 and x1

On the FVP platform BL3-1 params and BL3-1 plat params
and its constituents are stored at the start of TZDRAM.

The information about memory availability and size for
BL3-1, BL3-2 and BL3-3 is moved into platform specific data.

Change-Id: I8b32057a3d0dd3968ea26c2541a0714177820da9

At present, non-secure timer register contents are saved and restored as
part of world switch by BL3-1. This effectively means that the
non-secure timer stops, and non-secure timer interrupts are prevented
from asserting until BL3-1 switches back, introducing latency for
non-secure services. Often, secure world might depend on alternate
sources for secure interrupts (secure timer or platform timer) instead
of non-secure timers, in which case this save and restore is
unnecessary.

This patch introduces a boolean build-time configuration NS_TIMER_SWITCH
to choose whether or not to save and restore non-secure timer registers
upon world switch. The default choice is made not to save and restore
them.

Fixes ARM-software/tf-issues#148

Change-Id: I1b9d623606acb9797c3e0b02fb5ec7c0a414f37e

This patch implements the register reporting when unhandled exceptions are
taken in BL3-1. Unhandled exceptions will result in a dump of registers
to the console, before halting execution by that CPU. The Crash Stack,
previously called the Exception Stack, is used for this activity.
This stack is used to preserve the CPU context and runtime stack
contents for debugging and analysis.

This also introduces the per_cpu_ptr_cache, referenced by tpidr_el3,
to provide easy access to some of BL3-1 per-cpu data structures.
Initially, this is used to provide a pointer to the Crash stack.

panic() now prints the the error file and line number in Debug mode
and prints the PC value in release mode.

The Exception Stack is renamed to Crash Stack with this patch.
The original intention of exception stack is no longer valid
since we intend to support several valid exceptions like IRQ
and FIQ in the trusted firmware context. This stack is now
utilized for dumping and reporting the system state when a
crash happens and hence the rename.

Fixes ARM-software/tf-issues#79 Improve reporting of unhandled exception

Change-Id: I260791dc05536b78547412d147193cdccae7811a