GitLab

The WARMBOOT_ENABLE_DCACHE_EARLY allows caches to be turned on early during
the boot. But the xlat_change_mem_attributes_ctx() API did not do the required
cache maintenance after the mmap tables are modified if
WARMBOOT_ENABLE_DCACHE_EARLY is enabled. This meant that when the caches are turned
off during power down, the tables in memory are accessed as part of cache
maintenance for power down, and the tables are not correct at this point which
results in a data abort.
This patch removes the optimization within xlat_change_mem_attributes_ctx()
when WARMBOOT_ENABLE_DCACHE_EARLY is enabled.
Signed-off-by: Artsem Artsemenka <artsem.artsemenka@arm.com>
Change-Id: I82de3decba87dd13e9856b5f3620a1c8571c8d87

Coherent I-cache is causing a prefetch violation where when the core
executes an instruction that has recently been modified, the core might
fetch a stale instruction which violates the ordering of instruction
fetches.

The workaround includes an instruction sequence to implementation
defined registers to trap all EL0 IC IVAU instructions to EL3 and a trap
handler to execute a TLB inner-shareable invalidation to an arbitrary
address followed by a DSB.
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: Ic3b7cbb11cf2eaf9005523ef5578a372593ae4d6

Make the spinlock implementation use ARMv8.1-LSE CAS instruction based
on a platform build option. The CAS-based implementation used to be
unconditionally selected for all ARM8.1+ platforms.

The previous CAS spinlock implementation had a bug wherein the spin_unlock()
implementation had an `sev` after `stlr` which is not sufficient. A dsb is
needed to ensure that the stlr completes prior to the sev. Having a dsb is
heavyweight and a better solution would be to use load exclusive semantics
to monitor the lock and wake up from wfe when a store happens to the lock.
The patch implements the same.

Change-Id: I5283ce4a889376e4cc01d1b9d09afa8229a2e522
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
Signed-off-by: Olivier Deprez <olivier.deprez@arm.com>

This patch adds support for ARMv8.3-PAuth in BL1 SMC calls and
BL2U image for firmware updates by programming APIAKey_EL1 registers
and enabling Pointer Authentication in EL3 and EL1 respectively.

Change-Id: I875d952aba8242caf74fb5f4f2d2af6f0c768c08
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Change-Id: I1ea2bf088f1e001cdbd377cbfb7c6a2866af0422
Signed-off-by: Imre Kis <imre.kis@arm.com>

Change-Id: I645442d52a295706948e2cac88c36c1a3cb0bc47
Signed-off-by: Imre Kis <imre.kis@arm.com>

Not tested on FVP Model.

Change-Id: Iedebc5c1fbc7ea577e94142b7feafa5546f1f4f9
Signed-off-by: Artsem Artsemenka <artsem.artsemenka@arm.com>

This patch changes implementation for disabling Secure Cycle
Counter. For ARMv8.5 the counter gets disabled by setting
SDCR.SCCD bit on CPU cold/warm boot. For the earlier
architectures PMCR register is saved/restored on secure
world entry/exit from/to Non-secure state, and cycle counting
gets disabled by setting PMCR.DP bit.
In 'include\aarch32\arch.h' header file new
ARMv8.5-PMU related definitions were added.

Change-Id: Ia8845db2ebe8de940d66dff479225a5b879316f8
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

This PSCI hook is similar to pwr_domain_on_finish but is
guaranteed to be invoked with the respective core and cluster are
participating in coherency. This will be necessary to safely invoke
the new GICv3 API which modifies shared GIC data structures concurrently.

Change-Id: I8e54f05c9d4ef5712184c9c18ba45ac97a29eb7a
Signed-off-by: Madhukar Pappireddy <madhukar.pappireddy@arm.com>

assert() calls are removed in release builds, and if that assert call is
the only use of a variable, an unused variable warning will be triggered
in a release build. This patch fixes this problem when
CTX_INCLUDE_MTE_REGS by not using an intermediate variable to store the
results of get_armv8_5_mte_support().

Change-Id: I529e10ec0b2c8650d2c3ab52c4f0cecc0b3a670e
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

AArch64 System register SCTLR_EL1[31:0] is architecturally mapped
to AArch32 System register SCTLR[31:0]
AArch64 System register ACTLR_EL1[31:0] is architecturally mapped
to AArch32 System register ACTLR[31:0].

`u_register_t` should be used when it's important to store the
contents of a register in its native size
Signed-off-by: Deepika Bhavnani <deepika.bhavnani@arm.com>
Change-Id: I0055422f8cc0454405e011f53c1c4ddcaceb5779

This patch provides the following features and makes modifications
listed below:
- Individual APIAKey key generation for each CPU.
- New key generation on every BL31 warm boot and TSP CPU On event.
- Per-CPU storage of APIAKey added in percpu_data[]
  of cpu_data structure.
- `plat_init_apiakey()` function replaced with `plat_init_apkey()`
  which returns 128-bit value and uses Generic timer physical counter
  value to increase the randomness of the generated key.
  The new function can be used for generation of all ARMv8.3-PAuth keys
- ARMv8.3-PAuth specific code placed in `lib\extensions\pauth`.
- New `pauth_init_enable_el1()` and `pauth_init_enable_el3()` functions
  generate, program and enable APIAKey_EL1 for EL1 and EL3 respectively;
  pauth_disable_el1()` and `pauth_disable_el3()` functions disable
  PAuth for EL1 and EL3 respectively;
  `pauth_load_bl31_apiakey()` loads saved per-CPU APIAKey_EL1 from
  cpu-data structure.
- Combined `save_gp_pauth_registers()` function replaces calls to
  `save_gp_registers()` and `pauth_context_save()`;
  `restore_gp_pauth_registers()` replaces `pauth_context_restore()`
  and `restore_gp_registers()` calls.
- `restore_gp_registers_eret()` function removed with corresponding
  code placed in `el3_exit()`.
- Fixed the issue when `pauth_t pauth_ctx` structure allocated space
  for 12 uint64_t PAuth registers instead of 10 by removal of macro
  CTX_PACGAKEY_END from `include/lib/el3_runtime/aarch64/context.h`
  and assigning its value to CTX_PAUTH_REGS_END.
- Use of MODE_SP_ELX and MODE_SP_EL0 macro definitions
  in `msr	spsel`  instruction instead of hard-coded values.
- Changes in documentation related to ARMv8.3-PAuth and ARMv8.5-BTI.

Change-Id: Id18b81cc46f52a783a7e6a09b9f149b6ce803211
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

cpu_idx is used as mix of `unsigned int` and `signed int` in code
with typecasting at some places. This change is to unify the
cpu_idx as `unsigned int` as underlying API;s `plat_my_core_pos`
returns `unsigned int`

It was discovered via coverity issue CID 354715
Signed-off-by: Deepika Bhavnani <deepika.bhavnani@arm.com>
Change-Id: I4f0adb0c596ff1177210c5fe803bff853f2e54ce

Zeus supports the SSBS mechanism and also the new MSR instruction to
immediately apply the mitigation. Hence, the new instruction is utilised
in the Zeus-specific reset function.

Change-Id: I962747c28afe85a15207a0eba4146f9a115b27e7
Signed-off-by: John Tsichritzis <john.tsichritzis@arm.com>

Sparse warns this:

lib/libc/assert.c:29:6: error: symbol '__assert' redeclared with different type (originally declared at include/lib/libc/assert.h:36) - different modifiers

Add __dead2 to match the header declaration and C definition.

I also changed '__dead2 void' to 'void __dead2' for the consistency
with other parts.

Change-Id: Iefa4f0e787c24fa7e7e499d2e7baf54d4deb49ef
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>

Signed-off-by: Deepika Bhavnani <deepika.bhavnani@arm.com>
Change-Id: I4a496d5a8e7a9a127cd6224c968539eb74932fca

This patch adds support for the new Memory Tagging Extension arriving in
ARMv8.5. MTE support is now enabled by default on systems that support
at EL0. To enable it at ELx for both the non-secure and the secure
world, the compiler flag CTX_INCLUDE_MTE_REGS includes register saving
and restoring when necessary in order to prevent register leakage
between the worlds.

Change-Id: I2d4ea993d6b11654ea0d4757d00ca20d23acf36c
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

This patch fixes an issue when secure world timing information
can be leaked because Secure Cycle Counter is not disabled.
For ARMv8.5 the counter gets disabled by setting MDCR_El3.SCCD
bit on CPU cold/warm boot.
For the earlier architectures PMCR_EL0 register is saved/restored
on secure world entry/exit from/to Non-secure state, and cycle
counting gets disabled by setting PMCR_EL0.DP bit.
'include\aarch64\arch.h' header file was tided up and new
ARMv8.5-PMU related definitions were added.

Change-Id: I6f56db6bc77504634a352388990ad925a69ebbfa
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

ISB instruction was removed from the N1 1043202 Errata Workaround [1], this
fix is adding the ISB instruction back in.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html

Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>
Change-Id: I74eac7f6ad38991c36d423ad6aa44558033ad388

GCC diagnostics were added to ignore array boundaries, instead
of ignoring GCC warning current code will check for array boundaries
and perform and array update only for valid elements.

Resolves: `CID 246574` `CID 246710` `CID 246651`
Signed-off-by: Deepika Bhavnani <deepika.bhavnani@arm.com>
Change-Id: I7530ecf7a1707351c6ee87e90cc3d33574088f57

This patch fixes FVP_Base_AEMv8A model hang issue with
ARMv8.4+ with cache modelling enabled configuration.
Incorrect L1 cache flush operation to PoU, using CLIDR_EL1
LoUIS field, which is required by the architecture to be
zero for ARMv8.4-A with ARMv8.4-S2FWB feature is replaced
with L1 to L2 and L2 to L3 (if L3 is present) cache flushes.
FVP_Base_AEMv8A model can be configured with L3 enabled by
setting `cluster0.l3cache-size` and `cluster1.l3cache-size`
to non-zero values, and presence of L3 is checked in
`aem_generic_core_pwr_dwn` function by reading
CLIDR_EL1.Ctype3 field value.

Change-Id: If3de3d4eb5ed409e5b4ccdbc2fe6d5a01894a9af
Signed-off-by: Alexei Fedorov <Alexei.Fedorov@arm.com>

Coverity has identified that the __aeabi_imod function will loop forever
if the denominator is not a power of 2, which is probably not the
desired behaviour.

The functions in the rest of the file are compiler implementations of
division if ARMv7 does not implement division which is permitted by the
spec. However, while most of the functions in the file are documented
and referenced in other places online, __aeabi_uimod and __aeabi_imod
are not. For this reason, these functions have been removed from the
code base, which also removes the Coverity error.

Change-Id: I20066d72365329a8b03a5536d865c4acaa2139ae
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

This patch simply splits off the increment of next_xlat into a separate
statement to ensure consistent behaviour if the assert was to ever be
removed.

Change-Id: I827f601ccea55f4da9442048419c9b8cc0c5d22e
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

This ensures that probe_data starts with a reasonable default, as
opposed to whatever was left on the stack.

Change-Id: I5550efea5e2bec7717f9fa063cb11e6a7005cce5
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

NOTE: AARCH32/AARCH64 macros are now deprecated in favor of __aarch64__.

All common C compilers pre-define the same macros to signal which
architecture the code is being compiled for: __arm__ for AArch32 (or
earlier versions) and __aarch64__ for AArch64. There's no need for TF-A
to define its own custom macros for this. In order to unify code with
the export headers (which use __aarch64__ to avoid another dependency),
let's deprecate the AARCH32 and AARCH64 macros and switch the code base
over to the pre-defined standard macro. (Since it is somewhat
unintuitive that __arm__ only means AArch32, let's standardize on only
using __aarch64__.)

Change-Id: Ic77de4b052297d77f38fc95f95f65a8ee70cf200
Signed-off-by: Julius Werner <jwerner@chromium.org>

NOTE: __ASSEMBLY__ macro is now deprecated in favor of __ASSEMBLER__.

All common C compilers predefine a macro called __ASSEMBLER__ when
preprocessing a .S file. There is no reason for TF-A to define it's own
__ASSEMBLY__ macro for this purpose instead. To unify code with the
export headers (which use __ASSEMBLER__ to avoid one extra dependency),
let's deprecate __ASSEMBLY__ and switch the code base over to the
predefined standard.

Change-Id: Id7d0ec8cf330195da80499c68562b65cb5ab7417
Signed-off-by: Julius Werner <jwerner@chromium.org>

Change-Id: Ie0a94783d0c8e111ae19fd592304e6485f04ca29
Signed-off-by: Balint Dobszay <balint.dobszay@arm.com>

The features of the previously existing gentbl, genvar and genwrappers
scripts were reimplemented in the romlib_generator.py Python script.
This resulted in more readable and maintainable code and the script
introduces additional features that help dependency handling in
makefiles. The assembly templates were separated from the script logic
and were collected in the 'templates' directory.

The targets and their dependencies were reorganized in the makefile and
the dependency handling of included index files is possible now.
Incremental build is available in case of modifying the index files.
Signed-off-by: Imre Kis <imre.kis@arm.com>
Change-Id: I79f65fab9dc5c70d1f6fc8f57b2a3009bf842dc5

This patch adds some common helper code to support a lightweight
platform parameter passing framework between BLs that has already been
used on Rockchip platforms but is more widely useful to others as well.
It can be used as an implementation for the SoC firmware configuration
file mentioned in the docs, and is primarily intended for platforms
that only require a handful of values to be passed and want to get by
without a libfdt dependency. Parameters are stored in a linked list and
the parameter space is split in generic and vendor-specific parameter
types. Generic types will be handled by this code whereas
vendor-specific types have to be handled by a vendor-specific handler
function that gets passed in.

Change-Id: If3413d44e86b99d417294ce8d33eb2fc77a6183f
Signed-off-by: Julius Werner <jwerner@chromium.org>

Change-Id: Iab767e9937f5c6c8150953fcdc3b37e8ee83fa63
Signed-off-by: Louis Mayencourt <louis.mayencourt@arm.com>

This patch enables MTE for Normal world if the CPU suppors it. Enabling
MTE for secure world will be done later.

Change-Id: I9ef64460beaba15e9a9c20ab02da4fb2208b6f7d
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

This consists of ensuring that the left operand of each shift is
unsigned when the operation might overflow into the sign bit.

Change-Id: Iddd6f38139a4c6e500468b4fc48d04e0939f574e
Signed-off-by: Justin Chadwell <justin.chadwell@arm.com>

Change-Id: I755e4c42242d9a052570fd1132ca3d937acadb13
Signed-off-by: Balint Dobszay <balint.dobszay@arm.com>

Replacing ISB instructions in each Errata workaround with a single ISB
instruction before the RET in the reset handler.

Change-Id: I08afabc5b98986a6fe81664cd13822b36cab786f
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Neoverse N1 erratum 1275112 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR_EL1 system register, which delays instruction fetch after
branch misprediction.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html



Change-Id: If7fe55fe92e656fa6aea12327ab297f2e6119833
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Neoverse N1 erratum 1262888 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUECTLR_EL1 system register, which disables the MMU hardware prefetcher.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html



Change-Id: Ib733d748e32a7ea6a2783f3d5a9c5e13eee01105
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Neoverse N1 erratum 1262606 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR_EL1 system register, which delays instruction fetch after
branch misprediction.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html



Change-Id: Idd980e9d5310232d38f0ce272862e1fb0f02ce9a
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Neoverse N1 erratum 1257314 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR3_EL1 system register, which prevents parallel
execution of divide and square root instructions.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html



Change-Id: I54f0f40ff9043efee40d51e796b92ed85b394cbb
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Neoverse N1 erratum 1220197 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set two bits in the implementation defined
CPUECTLR_EL1 system register, which disables write streaming to the L2.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html



Change-Id: I9c3373f1b6d67d21ee71b2b80aec5e96826818e8
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>

Neoverse N1 erratum 1207823 is a Cat B erratum [1],
present in older revisions of the Neoverse N1 processor core.
The workaround is to set a bit in the implementation defined
CPUACTLR2_EL1 system register.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.pjdoc-466751330-10325/index.html



Change-Id: Ia932337821f1ef0d644db3612480462a8d924d21
Signed-off-by: Lauren Wehrmeister <lauren.wehrmeister@arm.com>