GitLab

EHF is a framework that allows dispatching of EL3 interrupts to their
respective handlers in EL3.

This framework facilitates the firmware-first error handling policy in
which asynchronous exceptions may be routed to EL3. Such exceptions may
be handed over to respective exception handlers. Individual handlers
might further delegate exception handling to lower ELs.

The framework associates the delegated execution to lower ELs with a
priority value. For interrupts, this corresponds to the priorities
programmed in GIC; for other types of exceptions, viz. SErrors or
Synchronous External Aborts, individual dispatchers shall explicitly
associate delegation to a secure priority. In order to prevent lower
priority interrupts from preempting higher priority execution, the
framework provides helpers to control preemption by virtue of
programming Priority Mask register in the interrupt controller.

This commit allows for handling interrupts targeted at EL3. Exception
handlers own interrupts by assigning them a range of secure priorities,
and registering handlers for each priority range it owns.

Support for exception handling in BL31 image is enabled by setting the
build option EL3_EXCEPTION_HANDLING=1.

Documentation to follow.

NOTE: The framework assumes the priority scheme supported by platform
interrupt controller is compliant with that of ARM GIC architecture (v2
or later).

Change-Id: I7224337e4cea47c6ca7d7a4ca22a3716939f7e42
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

This fix modifies the order of system includes to meet the ARM TF coding
standard. There are some exceptions in order to retain header groupings,
minimise changes to imported headers, and where there are headers within
the #if and #ifndef statements.

Change-Id: I65085a142ba6a83792b26efb47df1329153f1624
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/



Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Replace all instances of checks with the new macro.

Change-Id: I0eec39b9376475a1a9707a3115de9d36f88f8a2a
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Call console_flush() before execution either terminates or leaves an
exception level.

Fixes: ARM-software/tf-issues#123

Change-Id: I64eeb92effb039f76937ce89f877b68e355588e3
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

In order to quantify the overall time spent in the PSCI software
implementation, an initial collection of PMF instrumentation points
has been added.

Instrumentation has been added to the following code paths:

- Entry to PSCI SMC handler.  The timestamp is captured as early
  as possible during the runtime exception and stored in memory
  before entering the PSCI SMC handler.

- Exit from PSCI SMC handler.  The timestamp is captured after
  normal return from the PSCI SMC handler or if a low power state
  was requested it is captured in the bl31 warm boot path before
  return to normal world.

- Entry to low power state.  The timestamp is captured before entry
  to a low power state which implies either standby or power down.
  As these power states are mutually exclusive, only one timestamp
  is defined to describe both.  It is possible to differentiate between
  the two power states using the PSCI STAT interface.

- Exit from low power state.  The timestamp is captured after a standby
  or power up operation has completed.

To calculate the number of cycles spent running code in Trusted Firmware
one can perform the following calculation:

(exit_psci - enter_psci) - (exit_low_pwr - enter_low_pwr).

The resulting number of cycles can be converted to time given the
frequency of the counter.

Change-Id: Ie3b8f3d16409b6703747093b3a2d5c7429ad0166
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

This patch moves the invocation of `psci_setup()` from BL31 and SP_MIN
into `std_svc_setup()` as part of ARM Standard Service initialization.
This allows us to consolidate ARM Standard Service initializations which
will be added to in the future. A new function `get_arm_std_svc_args()`
is introduced to get arguments corresponding to each standard service.
This function must be implemented by the EL3 Runtime Firmware and both
SP_MIN and BL31 implement it.

Change-Id: I38e1b644f797fa4089b20574bd4a10f0419de184

This patch introduces a `psci_lib_args_t` structure which must be
passed into `psci_setup()` which is then used to initialize the PSCI
library. The `psci_lib_args_t` is a versioned structure so as to enable
compatibility checks during library initialization. Both BL31 and SP_MIN
are modified to use the new structure.

SP_MIN is also modified to add version string and build message as part
of its cold boot log just like the other BLs in Trusted Firmware.

NOTE: Please be aware that this patch modifies the prototype of
`psci_setup()`, which breaks compatibility with EL3 Runtime Firmware
(excluding BL31 and SP_MIN) integrated with the PSCI Library.

Change-Id: Ic3761db0b790760a7ad664d8a437c72ea5edbcd6

This patch introduces the PSCI Library interface. The major changes
introduced are as follows:

* Earlier BL31 was responsible for Architectural initialization during cold
boot via bl31_arch_setup() whereas PSCI was responsible for the same during
warm boot. This functionality is now consolidated by the PSCI library
and it does Architectural initialization via psci_arch_setup() during both
cold and warm boots.

* Earlier the warm boot entry point was always `psci_entrypoint()`. This was
not flexible enough as a library interface. Now PSCI expects the runtime
firmware to provide the entry point via `psci_setup()`. A new function
`bl31_warm_entrypoint` is introduced in BL31 and the previous
`psci_entrypoint()` is deprecated.

* The `smc_helpers.h` is reorganized to separate the SMC Calling Convention
defines from the Trusted Firmware SMC helpers. The former is now in a new
header file `smcc.h` and the SMC helpers are moved to Architecture specific
header.

* The CPU context is used by PSCI for context initialization and
restoration after power down (PSCI Context). It is also used by BL31 for SMC
handling and context management during Normal-Secure world switch (SMC
Context). The `psci_smc_handler()` interface is redefined to not use SMC
helper macros thus enabling to decouple the PSCI context from EL3 runtime
firmware SMC context. This enables PSCI to be integrated with other runtime
firmware using a different SMC context.

NOTE: With this patch the architectural setup done in `bl31_arch_setup()`
is done as part of `psci_setup()` and hence `bl31_platform_setup()` will be
invoked prior to architectural setup. It is highly unlikely that the platform
setup will depend on architectural setup and cause any failure. Please be
be aware of this change in sequence.

Change-Id: I7f497a08d33be234bbb822c28146250cb20dab73

The system registers that are saved and restored in CPU context include
AArch32 systems registers like SPSR_ABT, SPSR_UND, SPSR_IRQ, SPSR_FIQ,
DACR32_EL2, IFSR32_EL2 and FPEXC32_EL2. Accessing these registers on an
AArch64-only (i.e. on hardware that does not implement AArch32, or at
least not at EL1 and higher ELs) platform leads to an exception. This patch
introduces the build option `CTX_INCLUDE_AARCH32_REGS` to specify whether to
include these AArch32 systems registers in the cpu context or not. By default
this build option is set to 1 to ensure compatibility. AArch64-only platforms
must set it to 0. A runtime check is added in BL1 and BL31 cold boot path to
verify this.

Fixes ARM-software/tf-issues#386

Change-Id: I720cdbd7ed7f7d8516635a2ec80d025f478b95ee

This patch removes the dash character from the image name, to
follow the image terminology in the Trusted Firmware Wiki page:

    https://github.com/ARM-software/arm-trusted-firmware/wiki

Changes apply to output messages, comments and documentation.

non-ARM platform files have been left unmodified.

Change-Id: Ic2a99be4ed929d52afbeb27ac765ceffce46ed76

It is not ideal for BL31 to continue to use boot console at
runtime which could be potentially uninitialized. This patch
introduces a new optional platform porting API
`bl31_plat_runtime_setup()` which allows the platform to perform
any BL31 runtime setup just prior to BL31 exit during cold boot.
The default weak implementation of this function will invoke
`console_uninit()` which will suppress any BL31 runtime logs.

On the ARM Standard platforms, there is an anomaly that
the boot console will be reinitialized on resumption from
system suspend in `arm_system_pwr_domain_resume()`. This
will be resolved in the following patch.

NOTE: The default weak definition of `bl31_plat_runtime_setup()`
disables the BL31 console. To print the BL31 runtime
messages, platforms must override this API and initialize a
runtime console.

Fixes ARM-software/tf-issues#328

Change-Id: Ibaf8346fcceb447fe1a5674094c9f8eb4c09ac4a

This patch introduces a new function called 'print_entry_point_info'
that prints an entry_point_t structure for debugging purposes.
As such, it can be used to display the entry point address, SPSR and
arguments passed from a firmware image to the next one.

This function is now called in the following images transitions:
 - BL1 to BL2
 - BL1 to BL31
 - BL31 to the next image (typically BL32 or BL33)

The following changes have been introduced:

 - Fix the output format of the SPSR value : SPSR is a 32-bit value,
   not a 64-bit one.

 - Print all arguments values.
   The entry_point_info_t structure allows to pass up to 8 arguments.
   In most cases, only the first 2 arguments were printed.
   print_entry_point_info() now prints all of them as 'VERBOSE'
   traces.

Change-Id: Ieb384bffaa7849e6cb95a01a47c0b7fc2308653a

On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).

This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:

1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
   initialised. This ensures that any stale cache lines at any level of cache
   are removed.

2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
   visible to secondary CPUs using a cache clean operation by MVA.

3. Cache maintenance by set/way operations are only used prior to power down.

NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.

Fixes ARM-software/tf-issues#205

Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a

This patch migrates the rest of Trusted Firmware excluding Secure Payload and
the dispatchers to the new platform and context management API. The per-cpu
data framework APIs which took MPIDRs as their arguments are deleted and only
the ones which take core index as parameter are retained.

Change-Id: I839d05ad995df34d2163a1cfed6baa768a5a595d

Fix the following issues with the console log output:

* Make sure the welcome string is the first thing in the log output
(during normal boot).
* Prefix each message with the BL image name so it's clear which
BL the output is coming from.
* Ensure all output is wrapped in one of the log output macros so it can
be easily compiled out if necessary. Change some of the INFO() messages
to VERBOSE(), especially in the TSP.
* Create some extra NOTICE() and INFO() messages during cold boot.
* Remove all usage of \r in log output.

Fixes ARM-software/tf-issues#231

Change-Id: Ib24f7acb36ce64bbba549f204b9cde2dbb46c8a3

This patch adds support for BL3-2 initialization by asynchronous
method where BL3-1 transfers control to BL3-2 using world switch.
After BL3-2 initialization, it transfers control to BL3-3 via SPD
service handler. The SPD service handler initializes the CPU context
to BL3-3 entrypoint depending on the return function indentifier from
TSP initialization.

Fixes ARM-software/TF-issues#184

Change-Id: I7b135c2ceeb356d3bb5b6a287932e96ac67c7a34

Assert a valid security state using the macro sec_state_is_valid().
Replace assert() with panic() in those cases that might arise
because of runtime errors and not programming errors.
Replace panic() with assert() in those cases that might arise
because of programming errors.

Fixes ARM-software/tf-issues#96

Change-Id: I51e9ef0439fd5ff5e0edfef49050b69804bf14d5

Print out Trusted Firmware version at runtime at each BL stage.
Message consists of TF version as defined statically in the Makefile
(e.g. v0.4), build mode (debug|release) and a customizable build
string:

  1. By defining BUILD_STRING in command line when building TF
  2. Default string is git commit ID
  3. Empty if git meta-data is not available

Fixes ARM-software/tf-issues#203

Change-Id: I5c5ba438f66ab68810427d76b49c5b9177a957d6

This patch implements a "tf_printf" which supports only the commonly
used format specifiers in Trusted Firmware, which uses a lot less
stack space than the stdlib printf function.

Fixes ARM-software/tf-issues#116

Change-Id: I7dfa1944f4c1e634b3e2d571f49afe02d109a351

Exclude stdlib files because they do not follow kernel code style.

Fixes ARM-software/tf-issues#73

Change-Id: I4cfafa38ab436f5ab22c277cb38f884346a267ab

Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
*  The first uses entry_point_info to initialize the relevant
   cpu_context for first entry into a lower exception level on a CPU
*  The second populates the EL1 and EL2 system registers as needed
   from the cpu_context to ensure correct entry into the lower EL

This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).

In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.

Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f

The crash reporting support and early initialisation of the
cpu_data allow the runtime_exception vectors to be used from
the start in BL3-1, removing the need for the additional
early_exception vectors and 2KB of code from BL3-1.

Change-Id: I5f8997dabbaafd8935a7455910b7db174a25d871

This patch prepares the per-cpu pointer cache for wider use by:
* renaming the structure to cpu_data and placing in new header
* providing accessors for this CPU, or other CPUs
* splitting the initialization of the TPIDR pointer from the
  initialization of the cpu_data content
* moving the crash stack initialization to a crash stack function
* setting the TPIDR pointer very early during boot

Change-Id: Icef9004ff88f8eb241d48c14be3158087d7e49a3

All callers of cm_get_context() pass the calling CPU MPIDR to the
function. Providing a specialised version for the current
CPU results in a reduction in code size and better readability.

The current function has been renamed to cm_get_context_by_mpidr()
and the existing name is now used for the current-CPU version.

The same treatment has been done to cm_set_context(), although
only both forms are used at present in the PSCI and TSPD code.

Change-Id: I91cb0c2f7bfcb950a045dbd9ff7595751c0c0ffb

At present the arguments for BL3-3 in the entry_point_info
structure are not being transferred to X0-X7 before starting
execution of this image

This patch saves the args for BL3-3 into cpu context used
for its entry

Fixes ARM-software/tf-issues#172

Change-Id: I001b4b9bff6a264336f0d01d377619ae719f928b

Rename the ic_* platform porting functions to plat_ic_* to be
consistent with the other functions in platform.h. Also rename
bl31_get_next_image_info() to bl31_plat_get_next_image_ep_info()
and remove the duplicate declaration in bl31.h.

Change-Id: I4851842069d3cff14c0a468daacc0a891a7ede84

Some platform porting functions were in BL specific header files.
These have been moved to platform.h so that all porting functions
are in the same place. The functions are now grouped by BL.
Obsolete BL headers files have been removed.

Also, the weak declaration of the init_bl2_mem_layout() function
has been moved out the header file and into the source file
(bl_common.c) using the more succinct #pragma syntax. This
mitigates the risk of 2 weak definitions being created and the
wrong one being picked up by the compiler.

Change-Id: Ib19934939fd755f3e5a5a5bceec88da684308a83

The issues addressed in this patch are:

1. Remove meminfo_t from the common interfaces in BL3-x,
expecting that platform code will find a suitable mechanism
to determine the memory extents in these images and provide
it to the BL3-x images.

2. Remove meminfo_t and bl31_plat_params_t from all FVP BL3-x
code as the images use link-time information to determine
memory extents.

meminfo_t is still used by common interface in BL1/BL2 for
loading images

Change-Id: I4e825ebf6f515b59d84dc2bdddf6edbf15e2d60f

This patch is based on spec published at
https://github.com/ARM-software/tf-issues/issues/133

It rearranges the bl31_args struct into
bl31_params and bl31_plat_params which provide the
information needed for Trusted firmware and platform
specific data via x0 and x1

On the FVP platform BL3-1 params and BL3-1 plat params
and its constituents are stored at the start of TZDRAM.

The information about memory availability and size for
BL3-1, BL3-2 and BL3-3 is moved into platform specific data.

Change-Id: I8b32057a3d0dd3968ea26c2541a0714177820da9

SCR_EL3.RW was not updated immediately before exiting bl31_main() and
running BL3-3. If a AArch32 Secure-EL1 Payload had just been
initialised, then the SCR_EL3.RW bit would be left indicating a
32-bit BL3-3, which may not be correct.

This patch explicitly sets SCR_EL3.RW appropriately based on the
provided SPSR_EL3 value for the BL3-3 image.

Fixes ARM-software/tf-issues#126

Change-Id: Ic7716fe8bc87e577c4bfaeb46702e88deedd9895

This patch implements the register reporting when unhandled exceptions are
taken in BL3-1. Unhandled exceptions will result in a dump of registers
to the console, before halting execution by that CPU. The Crash Stack,
previously called the Exception Stack, is used for this activity.
This stack is used to preserve the CPU context and runtime stack
contents for debugging and analysis.

This also introduces the per_cpu_ptr_cache, referenced by tpidr_el3,
to provide easy access to some of BL3-1 per-cpu data structures.
Initially, this is used to provide a pointer to the Crash stack.

panic() now prints the the error file and line number in Debug mode
and prints the PC value in release mode.

The Exception Stack is renamed to Crash Stack with this patch.
The original intention of exception stack is no longer valid
since we intend to support several valid exceptions like IRQ
and FIQ in the trusted firmware context. This stack is now
utilized for dumping and reporting the system state when a
crash happens and hence the rename.

Fixes ARM-software/tf-issues#79 Improve reporting of unhandled exception

Change-Id: I260791dc05536b78547412d147193cdccae7811a

The current code does not always use data and instruction
barriers as required by the architecture and frequently uses
barriers excessively due to their inclusion in all of the
write_*() helper functions.

Barriers should be used explicitly in assembler or C code
when modifying processor state that requires the barriers in
order to enable review of correctness of the code.

This patch removes the barriers from the helper functions and
introduces them as necessary elsewhere in the code.

PORTING NOTE: check any port of Trusted Firmware for use of
system register helper functions for reliance on the previous
barrier behaviour and add explicit barriers as necessary.

Fixes ARM-software/tf-issues#92

Change-Id: Ie63e187404ff10e0bdcb39292dd9066cb84c53bf

Update code base to remove variables from the .data section,
mainly by using const static data where possible and adding
the const specifier as required. Most changes are to the IO
subsystem, including the framework APIs. The FVP power
management code is also affected.

Delay initialization of the global static variable,
next_image_type in bl31_main.c, until it is realy needed.
Doing this moves the variable from the .data to the .bss
section.

Also review the IO interface for inconsistencies, using
uintptr_t where possible instead of void *. Remove the
io_handle and io_dev_handle typedefs, which were
unnecessary, replacing instances with uintptr_t.

Fixes ARM-software/tf-issues#107.

Change-Id: I085a62197c82410b566e4698e5590063563ed304

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e

Add tag names to all unnamed structs in header files. This
allows forward declaration of structs, which is necessary to
reduce header file nesting (to be implemented in a subsequent
commit).

Also change the typedef names across the codebase to use the _t
suffix to be more conformant with the Linux coding style. The
coding style actually prefers us not to use typedefs at all but
this is considered a step too far for Trusted Firmware.

Also change the IO framework structs defintions to use typedef'd
structs to be consistent with the rest of the codebase.

Change-Id: I722b2c86fc0d92e4da3b15e5cab20373dd26786f

So it updates each time a bootloader changes, not just when bl*_main.c
files are recompiled.

Fixes ARM-software/tf-issues#33

Change-Id: Ie8e1a7bd7e1913d2e96ac268606284f76af8c5ab
Signed-off-by: Jon Medhurst <tixy@linaro.org>

At present SPD power management hooks and BL3-2 entry are implemented
using weak references. This would have the handlers bound and registered
with the core framework at build time, but leaves them dangling if a
service fails to initialize at runtime.

This patch replaces implementation by requiring runtime handlers to
register power management and deferred initialization hooks with the
core framework at runtime. The runtime services are to register the
hooks only as the last step, after having all states successfully
initialized.

Change-Id: Ibe788a2a381ef39aec1d4af5ba02376e67269782

This patch adds the following support to the BL3-1 stage:

1. BL3-1 allows runtime services to specify and determine the security
   state of the next image after BL3-1. This has been done by adding
   the `bl31_set_next_image_type()` & `bl31_get_next_image_type()`
   apis. The default security state is non-secure. The platform api
   `bl31_get_next_image_info()` has been modified to let the platform
   decide which is the next image in the desired security state.

2. BL3-1 exports the `bl31_prepare_next_image_entry()` function to
   program entry into the target security state. It uses the apis
   introduced in 1. to do so.

3. BL3-1 reads the information populated by BL2 about the BL3-2 image
   into its internal data structures.

4. BL3-1 introduces a weakly defined reference `bl32_init()` to allow
   initialisation of a BL3-2 image. A runtime service like the Secure
   payload dispatcher will define this function if present.

Change-Id: Icc46dcdb9e475ce6575dd3f9a5dc7a48a83d21d1

This patch uses the reworked exception handling support to handle
runtime service requests through SMCs following the SMC calling
convention. This is a giant commit since all the changes are
inter-related. It does the following:

1. Replace the old exception handling mechanism with the new one
2. Enforce that SP_EL0 is used C runtime stacks.
3. Ensures that the cold and warm boot paths use the 'cpu_context'
   structure to program an ERET into the next lower EL.
4. Ensures that SP_EL3 always points to the next 'cpu_context'
   structure prior to an ERET into the next lower EL
5. Introduces a PSCI SMC handler which completes the use of PSCI as a
   runtime service

Change-Id: I661797f834c0803d2c674d20f504df1b04c2b852
Co-authored-by: Achin Gupta <achin.gupta@arm.com>