GitLab

This patch fixes the bug when AMUv1 group1 counters was
always assumed being implemented without checking for its
presence which was causing exception otherwise.
The AMU extension code was also modified as listed below:
- Added detection of AMUv1 for ARMv8.6
- 'PLAT_AMU_GROUP1_NR_COUNTERS' build option is removed and
number of group1 counters 'AMU_GROUP1_NR_COUNTERS' is now
calculated based on 'AMU_GROUP1_COUNTERS_MASK' value
- Added bit fields definitions and access functions for
AMCFGR_EL0/AMCFGR and AMCGCR_EL0/AMCGCR registers
- Unification of amu.c Aarch64 and Aarch32 source files
- Bug fixes and TF-A coding style compliant changes.

Change-Id: I14e407be62c3026ebc674ec7045e240ccb71e1fb
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch adds all Tegra194 RAS nodes definitions and support to
handle all uncorrectable RAS errors.

Change-Id: I109b5a8dbca91d92752dc282c4ca30f273c475f9
Signed-off-by: David Pu <dpu@nvidia.com>
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

This patch fixes wrong ID_AA64DFR0_EL1 register read instead of
ID_AA64PFR0_EL1 to detect support for MPAM extension.
It also implements get_mpam_version() function which returns
MPAM version as:
0x00: None Armv8.0 or later;
0x01: v0.1 Armv8.4 or later;
0x10: v1.0 Armv8.2 or later;
0x11: v1.1 Armv8.4 or later;

Change-Id: I31d776b1a1b60cb16e5e62296d70adb129d7b760
Reported-by: Matteo Zini <matteozini96@gmail.com>
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch adds support for ARMv8.3-PAuth in BL1 SMC calls and
BL2U image for firmware updates by programming APIAKey_EL1 registers
and enabling Pointer Authentication in EL3 and EL1 respectively.

Change-Id: I875d952aba8242caf74fb5f4f2d2af6f0c768c08
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch provides the following features and makes modifications
listed below:
- Individual APIAKey key generation for each CPU.
- New key generation on every BL31 warm boot and TSP CPU On event.
- Per-CPU storage of APIAKey added in percpu_data[]
  of cpu_data structure.
- `plat_init_apiakey()` function replaced with `plat_init_apkey()`
  which returns 128-bit value and uses Generic timer physical counter
  value to increase the randomness of the generated key.
  The new function can be used for generation of all ARMv8.3-PAuth keys
- ARMv8.3-PAuth specific code placed in `lib\extensions\pauth`.
- New `pauth_init_enable_el1()` and `pauth_init_enable_el3()` functions
  generate, program and enable APIAKey_EL1 for EL1 and EL3 respectively;
  pauth_disable_el1()` and `pauth_disable_el3()` functions disable
  PAuth for EL1 and EL3 respectively;
  `pauth_load_bl31_apiakey()` loads saved per-CPU APIAKey_EL1 from
  cpu-data structure.
- Combined `save_gp_pauth_registers()` function replaces calls to
  `save_gp_registers()` and `pauth_context_save()`;
  `restore_gp_pauth_registers()` replaces `pauth_context_restore()`
  and `restore_gp_registers()` calls.
- `restore_gp_registers_eret()` function removed with corresponding
  code placed in `el3_exit()`.
- Fixed the issue when `pauth_t pauth_ctx` structure allocated space
  for 12 uint64_t PAuth registers instead of 10 by removal of macro
  CTX_PACGAKEY_END from `include/lib/el3_runtime/aarch64/context.h`
  and assigning its value to CTX_PAUTH_REGS_END.
- Use of MODE_SP_ELX and MODE_SP_EL0 macro definitions
  in `msr	spsel`  instruction instead of hard-coded values.
- Changes in documentation related to ARMv8.3-PAuth and ARMv8.5-BTI.

Change-Id: Id18b81cc46f52a783a7e6a09b9f149b6ce803211
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This ensures that probe_data starts with a reasonable default, as
opposed to whatever was left on the stack.

Change-Id: I5550efea5e2bec7717f9fa063cb11e6a7005cce5
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

This patch adds the functionality needed for platforms to provide
Branch Target Identification (BTI) extension, introduced to AArch64
in Armv8.5-A by adding BTI instruction used to mark valid targets
for indirect branches. The patch sets new GP bit [50] to the stage 1
Translation Table Block and Page entries to denote guarded EL3 code
pages which will cause processor to trap instructions in protected
pages trying to perform an indirect branch to any instruction other
than BTI.
BTI feature is selected by BRANCH_PROTECTION option which supersedes
the previous ENABLE_PAUTH used for Armv8.3-A Pointer Authentication
and is disabled by default. Enabling BTI requires compiler support
and was tested with GCC versions 9.0.0, 9.0.1 and 10.0.0.
The assembly macros and helpers are modified to accommodate the BTI
instruction.
This is an experimental feature.
Note. The previous ENABLE_PAUTH build option to enable PAuth in EL3
is now made as an internal flag and BRANCH_PROTECTION flag should be
used instead to enable Pointer Authentication.
Note. USE_LIBROM=1 option is currently not supported.

Change-Id: Ifaf4438609b16647dc79468b70cd1f47a623362e
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Complete the MPAM enablement in TF-A for lower ELs by enabling the EL2
traps in MPAMHCR_EL2 and MPAM2_EL2.This prevents an
MPAM-unaware-hypervisor to be restricted by an MPAM-aware-guest.

Change-Id: I47bf3f833fa22baa590f83d49cc0e3f2974e698d
Signed-off-by: Louis Mayencourt <louis.mayencourt@arm.com>

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca339

 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

No functional changes.

Change-Id: I2f28f20944f552447ac4e9e755493cd7c0ea1192
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Mark the initialization functions in BL31, such as context management,
EHF, RAS and PSCI as __init so that they can be reclaimed by the
platform when no longer needed

Change-Id: I7446aeee3dde8950b0f410cb766b7a2312c20130
Signed-off-by: Daniel Boulby <daniel.boulby@arm.com>

Change-Id: Ia02a2dbfd4e25547776e78bed40a91f3452553d7
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

These changes address most of the required MISRA rules. In the process,
some from generic code is also fixed.

No functional changes.

Change-Id: I76cacf6e1d73b09510561b5090c2bb66d81bec88
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Memory Partitioning And Monitoring is an Armv8.4 feature that enables
various memory system components and resources to define partitions.
Software running at various ELs can then assign themselves to the
desired partition to control their performance aspects.

With this patch, when ENABLE_MPAM_FOR_LOWER_ELS is set to 1, EL3 allows
lower ELs to access their own MPAM registers without trapping to EL3.
This patch however doesn't make use of partitioning in EL3; platform
initialisation code should configure and use partitions in EL3 if
required.

Change-Id: I5a55b6771ccaa0c1cffc05543d2116b60cbbcdcd
Co-authored-by: James Morse <james.morse@arm.com>
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Add a null pointer check for the error record probe
function -- avoids a panic in case a platform has not defined it.

Change-Id: I1139fa0df33297a12ec16615cacd07540925f991
Signed-off-by: Sughosh Ganu <sughosh.ganu@arm.com>

EHF currently allows for registering interrupt handlers for a defined
priority ranges. This is primarily targeted at various EL3 dispatchers
to own ranges of secure interrupt priorities in order to delegate
execution to lower ELs.

The RAS support added by earlier patches necessitates registering
handlers based on interrupt number so that error handling agents shall
receive and handle specific Error Recovery or Fault Handling interrupts
at EL3.

This patch introduces a macro, RAS_INTERRUPTS() to declare an array of
interrupt numbers and handlers. Error handling agents can use this macro
to register handlers for individual RAS interrupts. The array is
expected to be sorted in the increasing order of interrupt numbers.

As part of RAS initialisation, the list of all RAS interrupts are sorted
based on their ID so that, given an interrupt, its handler can be looked
up with a simple binary search.

For an error handling agent that wants to handle a RAS interrupt,
platform must:

  - Define PLAT_RAS_PRI to be the priority of all RAS exceptions.

  - Enumerate interrupts to have the GIC driver program individual EL3
    interrupts to the required priority range. This is required by EHF
    even before this patch.

Documentation to follow.

Change-Id: I9471e4887ff541f8a7a63309e9cd8f771f76aeda
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Previous patches added frameworks for handling RAS errors. This patch
introduces features that the platform can use to enumerate and iterate
RAS nodes:

  - The REGISTER_RAS_NODES() can be used to expose an array of
    ras_node_info_t structures. Each ras_node_info_t describes a RAS
    node, along with handlers for probing the node for error, and if
    did record an error, another handler to handle it.

  - The macro for_each_ras_node() can be used to iterate over the
    registered RAS nodes, probe for, and handle any errors.

The common platform EA handler has been amended using error handling
primitives introduced by both this and previous patches.

Change-Id: I2e13f65a88357bc48cd97d608db6c541fad73853
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

The ARMv8 RAS Extensions introduced Standard Error Records which are a
set of standard registers through which:

  - Platform can configure RAS node policy; e.g., notification
    mechanism;

  - RAS nodes can record and expose error information for error handling
    agents.

Standard Error Records can either be accessed via. memory-mapped
or System registers. This patch adds helper functions to access
registers and fields within an error record.

Change-Id: I6594ba799f4a1789d7b1e45b3e17fd40e7e0ba5c
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

Rule 8.4: A compatible declaration shall be visible when
          an object or function with external linkage is defined.

Change-Id: I26e042cb251a6f9590afa1340fdac73e42f23979
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

Change-Id: I38470528111410cf12b187eb1397d87b812c9416
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

AArch32 uses odd-even pairs when passing 64-bit arguments to
functions.  For example in `amu_group0_cnt_write_internal()` the
second argument is a uint64_t which is passed in r2 and r3.

In `amu_group1_set_evtype_internal()` the value that needs to be
written to the system register is in r1 not in r0.

Change-Id: I20196268fdb1dc9ef6c4ebe61e761fba9623b3f2
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Before suspend the AMU counters should be enabled and after resume
they should be disabled.  Assert that to be consistent with the
AArch64 implementation of `amu_context_{save,restore}()`.

Change-Id: Ia46f77e4062b93afb93721a2890a9b9d2a7f300e
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Implement helpers to test if the core supports SPE/SVE.  We have a
similar helper for AMU and this patch makes all extensions consistent
in their implementation.

Change-Id: I3e6f7522535ca358259ad142550b19fcb883ca67
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Add amu_context_save() and amu_context_restore() functions for aarch32

Change-Id: I4df83d447adeaa9d9f203e16dc5a919ffc04d87a
Signed-off-by: Joel Hutton <joel.hutton@arm.com>

Change-Id: Id6dfe885a63561b1d2649521bd020367b96ae1af
Signed-off-by: Joel Hutton <joel.hutton@arm.com>

If AMU is not supported by the hardware but it is enabled in Trusted
Firmware, the console will be spammed with warnings every time a CPU
is brought up with a CPU ON call.

Remove the warning message as this is more in line with how other
extensions like SPE and SVE are handled.

Change-Id: Iba6d367e4d1375ab554d23d2eaceab3ae1362c5a
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

On some systems, the AMU counters might reset to 0 when a CPU
powerdown happens.  This behaviour conflicts with the intended
use-case of AMU as lower ELs are only expected to see non-decreasing
counter values.

Change-Id: If25519965d4e6e47e09225d0e732947986cbb5ec
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Add some AMU helper functions to allow configuring, reading and
writing of the Group 0 and Group 1 counters.  Documentation for these
helpers will come in a separate patch.

Change-Id: I656e070d2dae830c22414f694aa655341d4e2c40
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

A new platform macro `PLAT_AMU_GROUP1_COUNTERS_MASK` controls which
group 1 counters should be enabled. The maximum number of group 1
counters supported by AMUv1 is 16 so the mask can be at most 0xffff.
If the platform does not define this mask, no group 1 counters are
enabled.

A related platform macro `PLAT_AMU_GROUP1_NR_COUNTERS` is used by
generic code to allocate an array to save and restore the counters on
CPU suspend.

Change-Id: I6d135badf4846292de931a43bb563077f42bb47b
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

This patch adds a new build option, ENABLE_SVE_FOR_NS, which when set
to one EL3 will check to see if the Scalable Vector Extension (SVE) is
implemented when entering and exiting the Non-secure world.

If SVE is implemented, EL3 will do the following:

- Entry to Non-secure world: SIMD, FP and SVE functionality is enabled.

- Exit from Non-secure world: SIMD, FP and SVE functionality is
  disabled. As SIMD and FP registers are part of the SVE Z-registers
  then any use of SIMD / FP functionality would corrupt the SVE
  registers.

The build option default is 1. The SVE functionality is only supported
on AArch64 and so the build option is set to zero when the target
archiecture is AArch32.

This build option is not compatible with the CTX_INCLUDE_FPREGS - an
assert will be raised on platforms where SVE is implemented and both
ENABLE_SVE_FOR_NS and CTX_INCLUDE_FPREGS are set to 1.

Also note this change prevents secure world use of FP&SIMD registers on
SVE-enabled platforms. Existing Secure-EL1 Payloads will not work on
such platforms unless ENABLE_SVE_FOR_NS is set to 0.

Additionally, on the first entry into the Non-secure world the SVE
functionality is enabled and the SVE Z-register length is set to the
maximum size allowed by the architecture. This includes the use case
where EL2 is implemented but not used.

Change-Id: Ie2d733ddaba0b9bef1d7c9765503155188fe7dae
Signed-off-by: David Cunado <david.cunado@arm.com>

The `ENABLE_AMU` build option can be used to enable the
architecturally defined AMU counters.  At present, there is no support
for the auxiliary counter group.

Change-Id: Ifc7532ef836f83e629f2a146739ab61e75c4abc8
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

The `ENABLE_AMU` build option can be used to enable the
architecturally defined AMU counters.  At present, there is no support
for the auxiliary counter group.

Change-Id: I7ea0c0a00327f463199d1b0a481f01dadb09d312
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Factor out SPE operations in a separate file.  Use the publish
subscribe framework to drain the SPE buffers before entering secure
world.  Additionally, enable SPE before entering normal world.

A side effect of this change is that the profiling buffers are now
only drained when a transition from normal world to secure world
happens.  Previously they were drained also on return from secure
world, which is unnecessary as SPE is not supported in S-EL1.

Change-Id: I17582c689b4b525770dbb6db098b3a0b5777b70a
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>