GitLab

This patch adds setting MDCR_EL3.MCCD in 'el3_arch_init_common'
macro to disable cycle counting by PMCCNTR_EL0 in EL3 when
FEAT_PMUv3p7 is implemented. This fixes failing test
'Leak PMU CYCLE counter values from EL3 on PSCI suspend SMC'
on FVP models with 'has_v8_7_pmu_extension' parameter set to
1 or 2.
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>
Change-Id: I2ad3ef501b31ee11306f76cb5a61032ecfd0fbda

Only BL32 (SP_min) is supported at the moment, BL1 and BL2_AT_EL3 are just
stubbed with _pie_fixup_size=0.
The changes are an adaptation for AARCH32 on what has been done for
PIE support on AARCH64.
The RELA_SECTION is redefined for AARCH32, as the created section is
.rel.dyn and the symbols are .rel*.

Change-Id: I92bafe70e6b77735f6f890f32f2b637b98cf01b9
Signed-off-by: Yann Gautier <yann.gautier@st.com>

The use of end addresses is preferred over the size of sections.
This was done for some AARCH64 files for PIE with commit [1],
and some extra explanations can be found in its commit message.
Align the missing AARCH64 files.

For AARCH32 files, this is required to prepare PIE support introduction.

 [1] f1722b69

 ("PIE: Use PC relative adrp/adr for symbol reference")

Change-Id: I8f1c06580182b10c680310850f72904e58a54d7d
Signed-off-by: Yann Gautier <yann.gautier@st.com>

The legacy console is gone. Re-add DCC console support based
on the multi-console framework.
Signed-off-by: Venkatesh Yadav Abbarapu <venkatesh.abbarapu@xilinx.com>
Acked-by: Michal Simek <michal.simek@xilinx.com>
Change-Id: Ia8388721093bc1be3af40974530d7c9a9ae5f43e

The speculation barrier feature (`FEAT_SB`) was introduced with and
made mandatory in the Armv8.5-A extension. It was retroactively made
optional in prior extensions, but the checks in our code-base do not
reflect that, assuming that it is only available in Armv8.5-A or later.

This change introduces the `ENABLE_FEAT_SB` definition, which derives
support for the `sb` instruction in the assembler from the feature
flags passed to it. Note that we assume that if this feature is enabled
then all the cores in the system support it - enabling speculation
barriers for only a subset of the cores is unsupported.
Signed-off-by: Chris Kay <chris.kay@arm.com>
Change-Id: I978ed38829385b221b10ba56d49b78f4756e20ea

ARMv8.6 adds virtual offset registers to support virtualization of the
event counters in EL1 and EL0.  This patch enables support for this
feature in EL3 firmware.
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I7ee1f3d9f554930bf5ef6f3d492e932e6d95b217

Add ISAR0 feature register read helper, location
of FEAT_RNG bits, feature support helper and the
rndr/rndrrs register read helpers.
Signed-off-by: Tomas Pilar <tomas@nuviainc.com>
Change-Id: I2a785a36f62a917548e55892ce92fa8b72fcb99d

If FEAT_PMUv3 is implemented and PMEVTYPER<n>(_EL0).MT bit is implemented
as well, it is possible to control whether PMU counters take into account
events happening on other threads.

If FEAT_MTPMU is implemented, EL3 (or EL2) can override the MT bit
leaving it to effective state of 0 regardless of any write to it.

This patch introduces the DISABLE_MTPMU flag, which allows to diable
multithread event count from EL3 (or EL2). The flag is disabled
by default so the behavior is consistent with those architectures
that do not implement FEAT_MTPMU.
Signed-off-by: Javier Almansa Sobrino <javier.almansasobrino@arm.com>
Change-Id: Iee3a8470ae8ba13316af1bd40c8d4aa86e0cb85e

This patch provides the following changes:
- Adds definition for FEAT_MTE3 value in ID_AA64PFR1_EL1 register
- Enables Memory Tagging Extension for FEAT_MTE3.

Change-Id: I735988575466fdc083892ec12c1aee89b5faa472
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch provides the changes listed below:
- Adds new bit fields definitions for SCTLR_EL1/2 registers
- Corrects the name of SCTLR_EL1/2.[20] bit field from
SCTLR_UWXN_BIT to SCTLR_TSCXT_BIT
- Adds FEAT_PANx bit field definitions and their possible
values for ID_AA64MMFR1_EL1 register.
- Adds setting of SCTLR_EL1.SPAN bit to preserve PSTATE.PAN
on taking an exception to EL1 in spm_sp_setup() function
(services\std_svc\spm_mm\spm_mm_setup.c)

Change-Id: If51f20e7995c649126a7728a4d0867041fdade19
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

When issuing barrier instructions like DSB or DMB, we must make sure
that the compiler does not undermine out efforts to fence off
instructions. Currently the compiler is free to move the barrier
instruction around, in respect to former or later memory access
statements, which is not what we want.

Add a compiler barrier to the inline assembly statement in our
DEFINE_SYSOP_TYPE_FUNC macro, to make sure memory accesses are not
reordered by the compiler.
This is in line with Linux' definition:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/include/asm/barrier.h



Since those instructions share a definition, apart from DSB and DMB this
now also covers some TLBI instructions. Having a compiler barrier there
also is useful, although we probably have stronger barriers in place
already.

Change-Id: If6fe97b13a562643a643efc507cb4aad29daa5b6
Reported-by: Alexandru Elisei <alexandru.elisei@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Usually, C has no problem up-converting types to larger bit sizes. MISRA
rule 10.7 requires that you not do this, or be very explicit about this.
This resolves the following required rule:

    bl1/aarch64/bl1_context_mgmt.c:81:[MISRA C-2012 Rule 10.7 (required)]<None>
    The width of the composite expression "0U | ((mode & 3U) << 2U) | 1U |
    0x3c0U" (32 bits) is less that the right hand operand
    "18446744073709547519ULL" (64 bits).

This also resolves MISRA defects such as:

    bl2/aarch64/bl2arch_setup.c:18:[MISRA C-2012 Rule 12.2 (required)]
    In the expression "3U << 20", shifting more than 7 bits, the number
    of bits in the essential type of the left expression, "3U", is
    not allowed.

Further, MISRA requires that all shifts don't overflow. The definition of
PAGE_SIZE was (1U << 12), and 1U is 8 bits. This caused about 50 issues.
This fixes the violation by changing the definition to 1UL << 12. Since
this uses 32bits, it should not create any issues for aarch32.

This patch also contains a fix for a build failure in the sun50i_a64
platform. Specifically, these misra fixes removed a single and
instruction,

    92407e73        and     x19, x19, #0xffffffff

from the cm_setup_context function caused a relocation in
psci_cpus_on_start to require a linker-generated stub. This increased the
size of the .text section and caused an alignment later on to go over a
page boundary and round up to the end of RAM before placing the .data
section. This sectionn is of non-zero size and therefore causes a link
error.

The fix included in this reorders the functions during link time
without changing their ording with respect to alignment.

Change-Id: I76b4b662c3d262296728a8b9aab7a33b02087f16
Signed-off-by: Jimmy Brisson <jimmy.brisson@arm.com>

As per latest mailing communication [1], we decided to
update AT speculative workaround implementation in order to
disable page table walk for lower ELs(EL1 or EL0) immediately
after context switching to EL3 from lower ELs.

Previous implementation of AT speculative workaround is available
here: 45aecff0

AT speculative workaround is updated as below:
1. Avoid saving and restoring of SCTLR and TCR registers for EL1
   in context save and restore routine respectively.
2. On EL3 entry, save SCTLR and TCR registers for EL1.
3. On EL3 entry, update EL1 system registers to disable stage 1
   page table walk for lower ELs (EL1 and EL0) and enable EL1
   MMU.
4. On EL3 exit, restore SCTLR and TCR registers for EL1 which
   are saved in step 2.

[1]:
https://lists.trustedfirmware.org/pipermail/tf-a/2020-July/000586.html



Change-Id: Iee8de16f81dc970a8f492726f2ddd57e7bd9ffb5
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

In case of AT speculative workaround applied, page table walk
is disabled for lower ELs (EL1 and EL0) in EL3.
Hence added a wrapper function which temporarily enables page
table walk to execute AT instruction for lower ELs and then
disables page table walk.

Execute AT instructions directly for lower ELs (EL1 and EL0)
assuming page table walk is enabled always when AT speculative
workaround is not applied.

Change-Id: I4ad4c0bcbb761448af257e9f72ae979473c0dde8
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>

This patch fixes the bug when AMUv1 group1 counters was
always assumed being implemented without checking for its
presence which was causing exception otherwise.
The AMU extension code was also modified as listed below:
- Added detection of AMUv1 for ARMv8.6
- 'PLAT_AMU_GROUP1_NR_COUNTERS' build option is removed and
number of group1 counters 'AMU_GROUP1_NR_COUNTERS' is now
calculated based on 'AMU_GROUP1_COUNTERS_MASK' value
- Added bit fields definitions and access functions for
AMCFGR_EL0/AMCFGR and AMCGCR_EL0/AMCGCR registers
- Unification of amu.c Aarch64 and Aarch32 source files
- Bug fixes and TF-A coding style compliant changes.

Change-Id: I14e407be62c3026ebc674ec7045e240ccb71e1fb
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch adds a build config 'RAS_TRAP_LOWER_EL_ERR_ACCESS' to set
SCR_EL3.TERR during CPU boot. This bit enables trapping RAS register
accesses from EL1 or EL2 to EL3.

RAS_TRAP_LOWER_EL_ERR_ACCESS is disabled by default.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: Ifb0fb0afedea7dd2a29a0b0491a1161ecd241438

Enhanced Counter Virtualization, ECV, is an architecture extension introduced
in ARMv8.6. This extension allows the hypervisor, at EL2, to setup
self-synchronizing views of the timers for it's EL1 Guests. This patch pokes the
control register to enable this extension when booting a hypervisor at EL2.

Change-Id: I4e929ecdf400cea17eff1de5cf8704aa7e40973d
Signed-off-by: Jimmy Brisson <jimmy.brisson@arm.com>

The Fine Grained Traps (FGT) architecture extension was added to aarch64 in
ARMv8.6. This extension primarily allows hypervisors, at EL2, to trap specific
instructions in a more fine grained manner, with an enable bit for each
instruction. This patch adds support for this extension by enabling the
extension when booting an hypervisor at EL2.

Change-Id: Idb9013ed118b6a1b7b76287237096de992ca4da3
Signed-off-by: Jimmy Brisson <jimmy.brisson@arm.com>

This patch fixes wrong ID_AA64DFR0_EL1 register read instead of
ID_AA64PFR0_EL1 to detect support for MPAM extension.
It also implements get_mpam_version() function which returns
MPAM version as:
0x00: None Armv8.0 or later;
0x01: v0.1 Armv8.4 or later;
0x10: v1.0 Armv8.2 or later;
0x11: v1.1 Armv8.4 or later;

Change-Id: I31d776b1a1b60cb16e5e62296d70adb129d7b760
Reported-by: Matteo Zini <matteozini96@gmail.com>
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch enables the v8.6 extension to add a delay before WFE traps
are taken. A weak hook plat_arm_set_twedel_scr_el3 has been added in
plat/common/aarch64/plat_common.c that disables this feature by default
but platform-specific code can override it when needed.

The only hook provided sets the TWED fields in SCR_EL3, there are similar
fields in HCR_EL2, SCTLR_EL2, and SCTLR_EL1 to control WFE trap delays in
lower ELs but these should be configured by code running at EL2 and/or EL1
depending on the platform configuration and is outside the scope of TF-A.
Signed-off-by: John Powell <john.powell@arm.com>
Change-Id: I0a9bb814205efeab693a3d0a0623e62144abba2d

This patch fixes compilation errors when ENABLE_PIE=1.

<snip>
bl31/aarch64/bl31_entrypoint.S: Assembler messages:
bl31/aarch64/bl31_entrypoint.S:61: Error: invalid operand (*UND* section) for `~'
bl31/aarch64/bl31_entrypoint.S:61: Error: invalid immediate
Makefile:1079: recipe for target 'build/tegra/t194/debug/bl31/bl31_entrypoint.o' failed
<snip>

Verified by setting 'ENABLE_PIE=1' for Tegra platform builds.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Change-Id: Ifd184f89b86b4360fda86a6ce83fd8495f930bbc

During context switching from higher EL (EL2 or higher)
to lower EL can cause incorrect translation in TLB due to
speculative execution of AT instruction using out-of-context
translation regime.

Workaround is implemented as below during EL's (EL1 or EL2)
"context_restore" operation:
1. Disable page table walk using SCTLR.M and TCR.EPD0 & EPD1
   bits for EL1 or EL2 (stage1 and stage2 disabled)
2. Save all system registers except TCR and SCTLR (for EL1 and EL2)
3. Do memory barrier operation (isb) to ensure all
   system register writes are done.
4. Restore TCR and SCTLR registers (for EL1 and EL2)

Errata details are available for various CPUs as below:
Cortex-A76: 1165522
Cortex-A72: 1319367
Cortex-A57: 1319537
Cortex-A55: 1530923
Cortex-A53: 1530924

More details can be found in mail-chain:
https://lists.trustedfirmware.org/pipermail/tf-a/2020-April/000445.html



Currently, Workaround is implemented as build option which is default
disabled.
Signed-off-by: Manish V Badarkhe <Manish.Badarkhe@arm.com>
Change-Id: If8545e61f782cb0c2dda7ffbaf50681c825bd2f0

As a follow-up to bdd2596d

, and related to SPM Dispatcher
EL3 component and SPM Core S-EL2/S-EL1 component: update
with cosmetic and coding rules changes. In addition:
-Add Armv8.4-SecEL2 arch detection helper.
-Add an SPMC context (on current core) get helper.
-Return more meaningful error return codes.
-Remove complexity in few spmd_smc_handler switch-cases.
-Remove unused defines and structures from spmd_private.h
Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Change-Id: I99e642450b0dafb19d3218a2f0e2d3107e8ca3fe

By writing 0 to CLUSTERPWRDN DSU register bit 0, we send an
advisory to the power controller that cluster power is not required
when all cores are powered down.

The AArch32 CLUSTERPWRDN register is architecturally mapped to the
AArch64 CLUSTERPWRDN_EL1 register

Change-Id: Ie6e67c1c7d811fa25c51e2e405ca7f59bd20c81b
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

bakery_lock_normal.c uses the raw register accessor, read_sctlr(_el3)
to check whether the dcache is enabled.

Using is_dcache_enabled() is cleaner, and a good abstraction for
the library code like this.

A problem is is_dcache_enabled() is declared in the local header,
lib/xlat_tables_v2/xlat_tables_private.h

I searched for a good place to declare this helper. Moving it to
arch_helpers.h, closed to cache operation helpers, looks good enough
to me.

I also changed the type of 'is_cached' to bool for consistency,
and to avoid MISRA warnings.

Change-Id: I9b016f67bc8eade25c316aa9c0db0fa4cd375b79
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Attempts to address MISRA compliance issues in BL1, BL2, and BL31 code.
Mainly issues like not using boolean expressions in conditionals,
conflicting variable names, ignoring return values without (void), adding
explicit casts, etc.

Change-Id: If1fa18ab621b9c374db73fa6eaa6f6e5e55c146a
Signed-off-by: John Powell <john.powell@arm.com>

There are some cases where we want to run EL-dependent code in the
shared code.

We could use #ifdef, but it leaves slight possibility where we do not
know the exception level at the build-time (e.g. library code).

The counter approach is to use get_current_el(), but it is run-time
detection, so all EL code is linked, some of which might be unneeded.

This commit adds get_current_el_maybe_constant(). This is a static
inline function that returns a constant value if we know the exception
level at build-time. This is mostly the case.

    if (get_current_el_maybe_constant() == 1) {
            /* do something for EL1 */
    } else if (get_current_el_maybe_constant() == 3) {
            /* do something for EL3 */
    }

If get_current_el_maybe_constant() is build-time constant, the compiler
will optimize out the unreachable code.

If such code is included from the library code, it is not built-time
constant. In this case, it falls back to get_current_el(), so it still
works.

Change-Id: Idb03c20342a5b5173fe2d6b40e1fac7998675ad3
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Change-Id: Ie1018bfbae2fe95c699e58648665baa75e862000
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

This patch provides a fix for incorrect crash dump data for
lower EL when TF-A is built with HANDLE_EA_EL3_FIRST=1 option
which enables routing of External Aborts and SErrors to EL3.

Change-Id: I9d5e6775e6aad21db5b78362da6c3a3d897df977
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This patch adds EL2 registers that are supported up to ARMv8.6.
ARM_ARCH_MINOR has to specified to enable save/restore routine.

Note: Following registers are still not covered in save/restore.
 * AMEVCNTVOFF0<n>_EL2
 * AMEVCNTVOFF1<n>_EL2
 * ICH_AP0R<n>_EL2
 * ICH_AP1R<n>_EL2
 * ICH_LR<n>_EL2

Change-Id: I4813f3243e56e21cb297b31ef549a4b38d4876e1
Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>

NOTE: Not all EL-2 system registers are saved/restored.
This subset includes registers recognized by ARMv8.0

Change-Id: I9993c7d78d8f5f8e72d1c6c8d6fd871283aa3ce0
Signed-off-by: Jose Marinho <jose.marinho@arm.com>
Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>
Signed-off-by: Artsem Artsemenka <artsem.artsemenka@arm.com>
Signed-off-by: Max Shvetsov <maksims.svecovs@arm.com>

aarch32 CPUs speculatively execute instructions following a
ERET as if it was not a jump instruction. This could lead to
cache-based side channel vulnerabilities. The software fix is
to place barrier instructions following ERET.

The counterpart patch for aarch64 is merged:
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/commit/?id=f461fe346b728d0e88142fd7b8f2816415af18bc



Change-Id: I2aa3105bee0b92238f389830b3a3b8650f33af3d
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

At each BL entry point, the registers r9 to r12 are used to save info from
the previous BL parameters put in r0 to r3. But zeromem uses r12, leading
to a corruption of arg3. Therefore this change copies r12 to r7 before
zeromem() call and restores r12 afterwards. It may be better to save it
in r7 in el3_arch_init_common and not at the entrypoint as r7 could be used
in other functions, especially platform ones.
This is a fix for Task T661.

Change-Id: Icc11990c69b5d4c542d08aca1a77b1f754b61a53
Signed-off-by: Yann Gautier <yann.gautier@st.com>

This patch modifies the delay timer driver to switch to the ARM
secure physical timer instead of using Tegra's on-chip uS timer.

The secure timer is not accessible to the NS world and so eliminates
an important attack vector, where the Tegra timer source gets switched
off from the NS world leading to a DoS attack for the trusted world.

This timer is shared with the S-EL1 layer for now, but later patches
will mark it as exclusive to the EL3 exception mode.

Change-Id: I2c00f8cb4c48b25578971c626c314603906ad7cc
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

At the moment, address demangling is only used by the backtrace
functionality. However, at some point, other parts of the TF-A
codebase may want to use it.
The 'demangle_address' function is replaced with a single XPACI
instruction which is also added in 'do_crash_reporting()'.
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>
Change-Id: I4424dcd54d5bf0a5f9b2a0a84c4e565eec7329ec

Even though ERET always causes a jump to another address, aarch64 CPUs
speculatively execute following instructions as if the ERET
instruction was not a jump instruction.
The speculative execution does not cross privilege-levels (to the jump
target as one would expect), but it continues on the kernel privilege
level as if the ERET instruction did not change the control flow -
thus execution anything that is accidentally linked after the ERET
instruction. Later, the results of this speculative execution are
always architecturally discarded, however they can leak data using
microarchitectural side channels. This speculative execution is very
reliable (seems to be unconditional) and it manages to complete even
relatively performance-heavy operations (e.g. multiple dependent
fetches from uncached memory).

This was fixed in Linux, FreeBSD, OpenBSD and Optee OS:
https://github.com/torvalds/linux/commit/679db70801da9fda91d26caf13bf5b5ccc74e8e8
https://github.com/freebsd/freebsd/commit/29fb48ace4186a41c409fde52bcf4216e9e50b61
https://github.com/openbsd/src/commit/3a08873ece1cb28ace89fd65e8f3c1375cc98de2
https://github.com/OP-TEE/optee_os/commit/abfd092aa19f9c0251e3d5551e2d68a9ebcfec8a

It is demonstrated in a SafeSide example:
https://github.com/google/safeside/blob/master/demos/eret_hvc_smc_wrapper.cc
https://github.com/google/safeside/blob/master/kernel_modules/kmod_eret_hvc_smc/eret_hvc_smc_module.c

Signed-off-by: Anthony Steinhauser <asteinhauser@google.com>
Change-Id: Iead39b0b9fb4b8d8b5609daaa8be81497ba63a0f

Some platforms are extremely memory constrained and must split BL31
between multiple non-contiguous areas in SRAM. Allow the NOBITS
sections (.bss, stacks, page tables, and coherent memory) to be placed
in a separate region of RAM from the loaded firmware image.

Because the NOBITS region may be at a lower address than the rest of
BL31, __RW_{START,END}__ and __BL31_{START,END}__ cannot include this
region, or el3_entrypoint_common would attempt to invalidate the dcache
for the entire address space. New symbols __NOBITS_{START,END}__ are
added when SEPARATE_NOBITS_REGION is enabled, and the dcached for the
NOBITS region is invalidated separately.
Signed-off-by: Samuel Holland <samuel@sholland.org>
Change-Id: Idedfec5e4dbee77e94f2fdd356e6ae6f4dc79d37

When a Firmware is complied as Position Independent Executable it needs
to request GDT fixup by passing size of the memory region to
el3_entrypoint_common macro.
The Global descriptor table fixup will be done early on during cold boot
process of primary core.

Currently only BL31 supports PIE, but in future when BL2_AT_EL3 will be
compiled as PIE, it can simply pass fixup size to the common el3
entrypoint macro to fixup GDT.

The reason for this patch was to overcome the bug introduced by SHA
330ead80

 which called fixup routine for each core causing
re-initializing of global pointers thus overwriting any changes
done by the previous core.

Change-Id: I55c792cc3ea9e7eef34c2e4653afd04572c4f055
Signed-off-by: Manish Pandey <manish.pandey2@arm.com>

Check that entry point information requesting S-EL2
has AArch64 as an execution state during context setup.
Signed-off-by: Artsem Artsemenka <artsem.artsemenka@arm.com>
Change-Id: I447263692fed6e55c1b076913e6eb73b1ea735b7

This patch adds support for enabling S-EL2 if this EL is specified in the entry
point information being used to initialise a secure context. It is the caller's
responsibility to check if S-EL2 is available on the system before requesting
this EL through the entry point information.
Signed-off-by: Achin Gupta <achin.gupta@arm.com>
Change-Id: I2752964f078ab528b2e80de71c7d2f35e60569e1