GitLab

If the CSV2 field reads as 1 then branch targets trained in one
context cannot affect speculative execution in a different context.
In that case skip the workaround on Cortex A75.

Change-Id: I4d5504cba516a67311fb5f0657b08f72909cbd38
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Invalidate the Branch Target Buffer (BTB) on entry to EL3 by
temporarily dropping into AArch32 Secure-EL1 and executing the
`BPIALL` instruction.

This is achieved by using 3 vector tables.  There is the runtime
vector table which is used to handle exceptions and 2 additional
tables which are required to implement this workaround.  The
additional tables are `vbar0` and `vbar1`.

The sequence of events for handling a single exception is
as follows:

1) Install vector table `vbar0` which saves the CPU context on entry
   to EL3 and sets up the Secure-EL1 context to execute in AArch32 mode
   with the MMU disabled and I$ enabled.  This is the default vector table.

2) Before doing an ERET into Secure-EL1, switch vbar to point to
   another vector table `vbar1`.  This is required to restore EL3 state
   when returning from the workaround, before proceeding with normal EL3
   exception handling.

3) While in Secure-EL1, the `BPIALL` instruction is executed and an
   SMC call back to EL3 is performed.

4) On entry to EL3 from Secure-EL1, the saved context from step 1) is
   restored.  The vbar is switched to point to `vbar0` in preparation to
   handle further exceptions.  Finally a branch to the runtime vector
   table entry is taken to complete the handling of the original
   exception.

This workaround is enabled by default on the affected CPUs.

NOTE
====

There are 4 different stubs in Secure-EL1.  Each stub corresponds to
an exception type such as Sync/IRQ/FIQ/SError.  Each stub will move a
different value in `R0` before doing an SMC call back into EL3.
Without this piece of information it would not be possible to know
what the original exception type was as we cannot use `ESR_EL3` to
distinguish between IRQs and FIQs.

Change-Id: I90b32d14a3735290b48685d43c70c99daaa4b434
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Invalidate the Branch Target Buffer (BTB) on entry to EL3 by disabling
and enabling the MMU.  To achieve this without performing any branch
instruction, a per-cpu vbar is installed which executes the workaround
and then branches off to the corresponding vector entry in the main
vector table.  A side effect of this change is that the main vbar is
configured before any reset handling.  This is to allow the per-cpu
reset function to override the vbar setting.

This workaround is enabled by default on the affected CPUs.

Change-Id: I97788d38463a5840a410e3cea85ed297a1678265
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

This patch adds a new build option, ENABLE_SVE_FOR_NS, which when set
to one EL3 will check to see if the Scalable Vector Extension (SVE) is
implemented when entering and exiting the Non-secure world.

If SVE is implemented, EL3 will do the following:

- Entry to Non-secure world: SIMD, FP and SVE functionality is enabled.

- Exit from Non-secure world: SIMD, FP and SVE functionality is
  disabled. As SIMD and FP registers are part of the SVE Z-registers
  then any use of SIMD / FP functionality would corrupt the SVE
  registers.

The build option default is 1. The SVE functionality is only supported
on AArch64 and so the build option is set to zero when the target
archiecture is AArch32.

This build option is not compatible with the CTX_INCLUDE_FPREGS - an
assert will be raised on platforms where SVE is implemented and both
ENABLE_SVE_FOR_NS and CTX_INCLUDE_FPREGS are set to 1.

Also note this change prevents secure world use of FP&SIMD registers on
SVE-enabled platforms. Existing Secure-EL1 Payloads will not work on
such platforms unless ENABLE_SVE_FOR_NS is set to 0.

Additionally, on the first entry into the Non-secure world the SVE
functionality is enabled and the SVE Z-register length is set to the
maximum size allowed by the architecture. This includes the use case
where EL2 is implemented but not used.

Change-Id: Ie2d733ddaba0b9bef1d7c9765503155188fe7dae
Signed-off-by: David Cunado <david.cunado@arm.com>

The `ENABLE_AMU` build option can be used to enable the
architecturally defined AMU counters.  At present, there is no support
for the auxiliary counter group.

Change-Id: Ifc7532ef836f83e629f2a146739ab61e75c4abc8
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

The `ENABLE_AMU` build option can be used to enable the
architecturally defined AMU counters.  At present, there is no support
for the auxiliary counter group.

Change-Id: I7ea0c0a00327f463199d1b0a481f01dadb09d312
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

The Cortex A75 has 5 AMU counters.  The first three counters are fixed
and the remaining two are programmable.

A new build option is introduced, `ENABLE_AMU`.  When set, the fixed
counters will be enabled for use by lower ELs.  The programmable
counters are currently disabled.

Change-Id: I4bd5208799bb9ed7d2596e8b0bfc87abbbe18740
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

This patch replaces the macro ASM_ASSERTION with the macro
ENABLE_ASSERTIONS in ARM Cortex-A53/57/72 MPCore Processor
related files. There is build error when ASM_ASSERTION is set
to 1 and ENABLE_ASSERTIONS is set to 0 because function
asm_assert in common/aarch32/debug.S is defined in the macro
ENABLE_ASSERTIONS but is called with the macro ASM_ASSERTION.

There is also the indication to use ENABLE_ASSERTIONS but not
ASM_ASSERTION in the Makefile.
Signed-off-by: Matt Ma <matt.ma@spreadtrum.com>

There is an edge case where the cache maintaince done in
psci_do_cpu_off may not seen by some cores. This case is handled in
psci_cpu_on_start but it hasn't handled in psci_affinity_info.

Change-Id: I4d64f3d1ca9528e364aea8d04e2d254f201e1702
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

Factor out SPE operations in a separate file.  Use the publish
subscribe framework to drain the SPE buffers before entering secure
world.  Additionally, enable SPE before entering normal world.

A side effect of this change is that the profiling buffers are now
only drained when a transition from normal world to secure world
happens.  Previously they were drained also on return from secure
world, which is unnecessary as SPE is not supported in S-EL1.

Change-Id: I17582c689b4b525770dbb6db098b3a0b5777b70a
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

Factor out extension enabling to a separate function that is called
before exiting from EL3 for first entry into Non-secure world.

Change-Id: Ic21401ebba531134d08643c0a1ca9de0fc590a1b
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

If an implementation of ARMv8.2 includes ARMv8.2-LPA, the value 0b0110
is permitted in ID_AA64MMFR0_EL1.PARange, which means that the Physical
Address range supported is 52 bits (4 PiB). It is a reserved value
otherwise.

Change-Id: Ie0147218e9650aa09f0034a9ee03c1cca8db908a
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

The FPEXC32_EL2 register controls SIMD and FP functionality when the
lower ELs are executing in AArch32 mode. It is architecturally mapped
to AArch32 system register FPEXC.

This patch removes FPEXC32_EL2 register from the System Register context
and adds it to the floating-point context. EL3 only saves / restores the
floating-point context if the build option CTX_INCLUDE_FPREGS is set to 1.

The rationale for this change is that if the Secure world is using FP
functionality and EL3 is not managing the FP context, then the Secure
world will save / restore the appropriate FP registers.

NOTE - this is a break in behaviour in the unlikely case that
CTX_INCLUDE_FPREGS is set to 0 and the platform contains an AArch32
Secure Payload that modifies FPEXC, but does not save and restore
this register

Change-Id: Iab80abcbfe302752d52b323b4abcc334b585c184
Signed-off-by: David Cunado <david.cunado@arm.com>

This function can be useful to setup TCR_ELx by callers that don't use
the translation tables library to setup the system registers related
to them. By making it common, it can be reused whenever it is needed
without duplicating code.

Change-Id: Ibfada9e846d2a6cd113b1925ac911bb27327d375
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

ARMv7-A architectures that do not support the Virtualization extensions
do not support instructions for the 32bit division. This change provides
a software implementation for 32bit division.

The division implementation is dumped from the OP-TEE project
http://github.com/OP-TEE/optee_os

. The code was slightly modified
to pass trusted firmware checkpatch requirements and copyright is
given to the ARM trusted firmware initiative and its contributors.

Change-Id: Idae0c7b80a0d75eac9bd41ae121921d4c5af3fa3
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

If ARMv7 based platform does not set ARM_CORTEX_Ax=yes, platform
shall define ARMV7_SUPPORTS_GENERIC_TIMER to enable generic timer
support.
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

ARCH_SUPPORTS_LARGE_PAGE_ADDRESSING allows build environment to
handle specific case when target ARMv7 core only supports 32bit MMU
descriptor mode.

If ARMv7 based platform does not set ARM_CORTEX_Ax=yes, platform
shall define ARMV7_SUPPORTS_LARGE_PAGE_ADDRESSING to enable
large page addressing support.
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

As Cortex-A9 needs to manually enable program flow prediction,
do not reset SCTLR[Z] at entry. Platform should enable it only
once MMU is enabled.

Change-Id: I34e1ee2da73221903f7767f23bc6fc10ad01e3de
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

Also need to add a SEV instruction in ARMv7 spin_unlock which
is implicit in ARMv8.
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

This patch includes various fixes for PSCI STAT functionality
relating to timestamp collection:

1. The PSCI stat accounting for retention states for higher level
power domains were done outside the locks which could lead to
spurious values in some race conditions. This is moved inside
the locks. Also, the call to start the stat accounting was redundant
which is now removed.

2. The timestamp wrap-around case when calculating residency did
not cater for AArch32. This is now fixed.

3. In the warm boot path, `plat_psci_stat_accounting_stop()` was
getting invoked prior to population of target power states. This
is now corrected.

Change-Id: I851526455304fb74ff0a724f4d5318cd89e19589
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

Add events that trigger before entry to normal/secure world.  The
events trigger after the normal/secure context has been restored.

Similarly add events that trigger after leaving normal/secure world.
The events trigger after the normal/secure context has been saved.

Change-Id: I1b48a7ea005d56b1f25e2b5313d77e67d2f02bc5
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

This allows other EL3 components to subscribe to CPU on events.

Update Firmware Design guide to list psci_cpu_on_finish as an available
event.

Change-Id: Ida774afe0f9cdce4021933fcc33a9527ba7aaae2
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

The implementation is the same as those used to disable it in EL3.

Change-Id: Ibfe7e69034a691fbf57477c5a76a8cdca28f6b26
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This patch introduces a new API in the translation tables library
(v2), that allows to change the memory attributes of a memory
region. It may be used to change its execution permissions and
data access permissions.

As a prerequisite, the memory must be already mapped. Moreover, it
must be mapped at the finest granularity (currently 4 KB).

Change-Id: I242a8c6f0f3ef2b0a81a61e28706540462faca3c
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This patch introduces a new API in the translation tables library
(v2), that allows to query the memory attributes of a memory block
or a memory page.

Change-Id: I45a8b39a53da39e7617cbac4bff5658dc1b20a11
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

With this patch the PSCI_FEATURES API correctly reports availability
of the PSCI_MEM_PROTECT_CHECK API - PSCI_MEM_CHK_RANGE_AARCH64 is
added to the PSCI capabilities mask, PSCI_CAP_64BIT_MASK

Change-Id: Ic90ee804deaadf0f948dc2d46ac5fe4121ef77ae
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

Currently TF does not initialise the PMCR_EL0 register in
the secure context or save/restore the register.

In particular, the DP field may not be set to one to prohibit
cycle counting in the secure state, even though event counting
generally is prohibited via the default setting of MDCR_EL3.SMPE
to 0.

This patch initialises PMCR_EL0.DP to one in the secure state
to prohibit cycle counting and also initialises other fields
that have an architectually UNKNOWN reset value.

Additionally, PMCR_EL0 is added to the list of registers that are
saved and restored during a world switch.

Similar changes are made for PMCR for the AArch32 execution state.

NOTE: secure world code at lower ELs that assume other values in PMCR_EL0
will be impacted.

Change-Id: Iae40e8c0a196d74053accf97063ebc257b4d2f3a
Signed-off-by: David Cunado <david.cunado@arm.com>

This patch implements PSCI_SYSTEM_RESET2 API as defined in PSCI
v1.1 specification. The specification allows architectural and
vendor-specific resets via this API. In the current specification,
there is only one architectural reset, the warm reset. This reset is
intended to provide a fast reboot path that guarantees not to reset
system main memory.

Change-Id: I057bb81a60cd0fe56465dbb5791d8e1cca025bd3
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

This patch introduces the ability of the xlat tables library to manage
EL0 and EL1 mappings from a higher exception level.

Attributes MT_USER and MT_PRIVILEGED have been added to allow the user
specify the target EL in the translation regime EL1&0.

REGISTER_XLAT_CONTEXT2 macro is introduced to allow creating a
xlat_ctx_t that targets a given translation regime (EL1&0 or EL3).

A new member is added to xlat_ctx_t to represent the translation regime
the xlat_ctx_t manages. The execute_never mask member is removed as it
is computed from existing information.

Change-Id: I95e14abc3371d7a6d6a358cc54c688aa9975c110
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Introduce a variant of the TLB invalidation helper function that
allows the targeted translation regime to be specified, rather than
defaulting to the current one.

This new function is useful in the context of EL3 software managing
translation tables for the S-EL1&0 translation regime, as then it
might need to invalidate S-EL1&0 TLB entries rather than EL3 ones.

Define a new enumeration to be able to represent translation regimes in
the xlat tables library.

Change-Id: Ibe4438dbea2d7a6e7470bfb68ff805d8bf6b07e5
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

TLB invalidation functions used to be conditionally compiled in.
They were enabled only when using the dynamic mapping feature.
because only then would we need to modify page tables on the fly.

Actually there are other use cases where invalidating TLBs is required.
When changing memory attributes in existing translation descriptors for
example. These other use cases do not necessarily depend on the dynamic
mapping feature.

This patch removes this dependency and always compile TLB invalidation
functions in. If they're not used, they will be removed from the binary
at link-time anyway so there's no consequence on the memory footprint
if these functions are not called.

Change-Id: I1c33764ae900eb00073ee23b7d0d53d4efa4dd21
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

The current implementation of the memory mapping API favours mapping
memory regions using the biggest possible block size in order to
reduce the number of translation tables needed.

In some cases, this behaviour might not be desirable. When translation
tables are edited at run-time, coarse-grain mappings like that might
need splitting into finer-grain tables. This operation has a
performance cost.

The MAP_REGION2() macro allows to specify the granularity of
translation tables used for the initial mapping of a memory region.
This might increase performance for memory regions that are likely to
be edited in the future, at the expense of a potentially increased
memory footprint.

The Translation Tables Library Design Guide has been updated to
explain the use case for this macro. Also added a few intermediate
titles to make the guide easier to digest.

Change-Id: I04de9302e0ee3d326b8877043a9f638766b81b7b
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This commit introduces a new type (mem_region_t) used to describe
memory regions and it adds two utility functions:

	- clear_mem_regions: This function clears (write 0) to a set
		of regions described with an array of mem_region_t.

	- mem_region_in_array_chk This function checks if a
		region is covered by some of the regions described
		with an array of mem_region_t.

Change-Id: I12ce549f5e81dd15ac0981645f6e08ee7c120811
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

This patch adds the generic code that links the psci smc handler
with the platform function that implements the mem_protect and
mem_check_range functionalities. These functions are  optional
APIs added in PSCI v1.1 (ARM DEN022D).

Change-Id: I3bac1307a5ce2c7a196ace76db8317e8d8c8bb3f
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

In the S-EL1&0 translation regime we aren't using the higher VA range,
whose translation table base address is held in TTBR1_EL1. The bit
TCR_EL1.EPD1 can be used to disable translations using TTBR1_EL1, but
the code wasn't setting it to 1. Additionally, other fields in TCR1_EL1
associated with the higher VA range (TBI1, TG1, SH1, ORGN1, IRGN1 and
A1) weren't set correctly as they were left as 0. In particular, 0 is a
reserved value for TG1. Also, TBBR1_EL1 was not explicitly set and its
reset value is UNKNOWN.

Therefore memory accesses to the higher VA range would result in
unpredictable behaviour as a translation table walk would be attempted
using an UNKNOWN value in TTBR1_EL1.

On the FVP and Juno platforms accessing the higher VA range resulted in
a translation fault, but this may not always be the case on all
platforms.

This patch sets the bit TCR_EL1.EPD1 to 1 so that any kind of
unpredictable behaviour is prevented.

This bug only affects the AArch64 version of the code, the AArch32
version sets this bit to 1 as expected.

Change-Id: I481c000deda5bc33a475631301767b9e0474a303
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>