GitLab

During cold boot, the initial translation tables are created with data
caches disabled, so all modifications go to memory directly. After the
MMU is enabled and data cache is enabled, any modification to the tables
goes to data cache, and eventually may get flushed to memory.

If CPU0 modifies the tables while CPU1 is off, CPU0 will have the
modified tables in its data cache. When CPU1 is powered on, the MMU is
enabled, then it enables coherency, and then it enables the data cache.
Until this is done, CPU1 isn't in coherency, and the translation tables
it sees can be outdated if CPU0 still has some modified entries in its
data cache.

This can be a problem in some cases. For example, the warm boot code
uses only the tables mapped during cold boot, which don't normally
change. However, if they are modified (and a RO page is made RW, or a XN
page is made executable) the CPU will see the old attributes and crash
when it tries to access it.

This doesn't happen in systems with HW_ASSISTED_COHERENCY or
WARMBOOT_ENABLE_DCACHE_EARLY. In these systems, the data cache is
enabled at the same time as the MMU. As soon as this happens, the CPU is
in coherency.

There was an attempt of a fix in psci_helpers.S, but it didn't solve the
problem. That code has been deleted. The code was introduced in commit
<26441030

> ("Invalidate TLB entries during warm boot").

Now, during a map or unmap operation, the memory associated to each
modified table is flushed. Traversing a table will also flush it's
memory, as there is no way to tell in the current implementation if the
table that has been traversed has also been modified.

Change-Id: I4b520bca27502f1018878061bc5fb82af740bb92
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Change-Id: I35d5b6a7c219f6f38983b30f157c1ed3808af17f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix defects of MISRA C-2012 rules 8.13, 10.1, 10.3, 10.4, 10.8, 11.6,
14.4, 15.7, 17.8, 20.10, 20.12, 21.1 and Directive 4.9.

Change-Id: I7ff61e71733908596dbafe2e99d99b4fce9765bd
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This library can be used in other projects. All comments that talk about
the Trusted Firmware should be talking about the library itself.

Change-Id: I3b98d42f7132be72c1f8a4900acfaa78dbd2daa2
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This allows other parts of the code to reuse it. No functional changes.

Change-Id: Ib052ae235c422d9179958bd3016c3e678779ae9b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Most registers are 64-bit wide, even in AArch32 mode:

- MAIR_ELx is equivalent to MAIR0 and MAIR1.
- TTBR is 64 bit in both AArch64 and AArch32.

The only difference is the TCR register, which is 32 bit in AArch32 and
in EL3 in AArch64. For consistency with the rest of ELs in AArch64, it
makes sense to also have it as a 64-bit value.

Change-Id: I2274d66a28876702e7085df5f8aad0e7ec139da9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

The Exception Level is now detected at runtime. This means that it is not
needed to hardcode the EL used by each image.

This doesn't result in a substantial increase of the image size because
the initialization functions that aren't used are garbage-collected by
the linker.

In AArch32 the current EL has been changed from EL3 to EL1 because the
the AArch32 PL1&0 translation regime behaves more like the AArch64 EL1&0
translation regime than the EL3 one.

Change-Id: I941404299ebe7666ca17619207c923b49a55cb73
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

xlat_arch_tlbi_va_regime() isn't used, so it has been renamed to
xlat_arch_tlbi_va() and the previous implementation has been removed.

Change-Id: Ic118bed3fb68234748d86b2e9e95b25650289276
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

They only contained one function that is easily integrated in the private
library header and the existing architectural C files.

This also helps making the library more portable, as the Makefile of the
library now doesn't use the variable INCLUDES, which is specific to this
codebase and doesn't respect the namespace of the library.

Change-Id: I22228e6a97e9b4f346f5cd8947609263e8df71d8
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Instead of having one big file with all the code, it's better to have
a few smaller files that are more manageable:

- xlat_tables_core.c: Code related to the core functionality of the
  library (map and unmap regions, initialize xlat context).
- xlat_tables_context.c: Instantiation of the active image context
  as well as APIs to manipulate it.
- xlat_tables_utils.c: Helper code that isn't part of the core
  functionality (change attributes, debug print messages).

Change-Id: I3ea956fc1afd7473c0bb5e7c6aab3b2e5d88c711
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

At present, the function provided by the translation library to enable
MMU constructs appropriate values for translation library, and programs
them to the right registers. The construction of initial values,
however, is only required once as both the primary and secondaries
program the same values.

Additionally, the MMU-enabling function is written in C, which means
there's an active stack at the time of enabling MMU. On some systems,
like Arm DynamIQ, having active stack while enabling MMU during warm
boot might lead to coherency problems.

This patch addresses both the above problems by:

  - Splitting the MMU-enabling function into two: one that sets up
    values to be programmed into the registers, and another one that
    takes the pre-computed values and writes to the appropriate
    registers. With this, the primary effectively calls both functions
    to have the MMU enabled, but secondaries only need to call the
    latter.

  - Rewriting the function that enables MMU in assembly so that it
    doesn't use stack.

This patch fixes a bunch of MISRA issues on the way.

Change-Id: I0faca97263a970ffe765f0e731a1417e43fbfc45
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

The function xlat_arch_is_granule_size_supported() can be used to check
if a specific granule size is supported. In Armv8, AArch32 only supports
4 KiB pages. AArch64 supports 4 KiB, 16 KiB or 64 KiB depending on the
implementation, which is detected at runtime.

The function xlat_arch_get_max_supported_granule_size() returns the max
granule size supported by the implementation.

Even though right now they are only used by SPM, they may be useful in
other places in the future. This patch moves the code currently in SPM
to the xlat tables lib so that it can be reused.

Change-Id: If54624a5ecf20b9b9b7f38861b56383a03bbc8a4
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

MISRA C-2012 Rule 7.3 violation: lowercase l shall not be used as literal suffixes.

This patch resolves this for the ULL() macro by using ULL suffix instead
of the ull suffix.

Change-Id: Ia8183c399e74677e676956e8653e82375d0e0a01
Signed-off-by: David Cunado <david.cunado@arm.com>

ARCH_SUPPORTS_LARGE_PAGE_ADDRESSING allows build environment to
handle specific case when target ARMv7 core only supports 32bit MMU
descriptor mode.

If ARMv7 based platform does not set ARM_CORTEX_Ax=yes, platform
shall define ARMV7_SUPPORTS_LARGE_PAGE_ADDRESSING to enable
large page addressing support.
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

This patch introduces a new API in the translation tables library
(v2), that allows to change the memory attributes of a memory
region. It may be used to change its execution permissions and
data access permissions.

As a prerequisite, the memory must be already mapped. Moreover, it
must be mapped at the finest granularity (currently 4 KB).

Change-Id: I242a8c6f0f3ef2b0a81a61e28706540462faca3c
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This patch introduces the ability of the xlat tables library to manage
EL0 and EL1 mappings from a higher exception level.

Attributes MT_USER and MT_PRIVILEGED have been added to allow the user
specify the target EL in the translation regime EL1&0.

REGISTER_XLAT_CONTEXT2 macro is introduced to allow creating a
xlat_ctx_t that targets a given translation regime (EL1&0 or EL3).

A new member is added to xlat_ctx_t to represent the translation regime
the xlat_ctx_t manages. The execute_never mask member is removed as it
is computed from existing information.

Change-Id: I95e14abc3371d7a6d6a358cc54c688aa9975c110
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Introduce a variant of the TLB invalidation helper function that
allows the targeted translation regime to be specified, rather than
defaulting to the current one.

This new function is useful in the context of EL3 software managing
translation tables for the S-EL1&0 translation regime, as then it
might need to invalidate S-EL1&0 TLB entries rather than EL3 ones.

Define a new enumeration to be able to represent translation regimes in
the xlat tables library.

Change-Id: Ibe4438dbea2d7a6e7470bfb68ff805d8bf6b07e5
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This patch enables the CnP (Common not Private) bit for secure page
tables so that multiple PEs in the same Inner Shareable domain can use
the same translation table entries for a given stage of translation in
a particular translation regime. This only takes effect when ARM
Trusted Firmware is built with ARM_ARCH_MINOR >= 2.

ARM Trusted Firmware Design has been updated to include a description
of this feature usage.

Change-Id: I698305f047400119aa1900d34c65368022e410b8
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>

Fix the type length and signedness of some of the constants and
variables used in the translation table library.

This patch supersedes Pull Request #1018:
https://github.com/ARM-software/arm-trusted-firmware/pull/1018



Change-Id: Ibd45faf7a4fb428a0bf71c752551d35800212fb2
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

Previous patches have made it possible to specify the physical and
virtual address spaces sizes for each translation context. However,
there are still some places in the code where the physical (resp.
virtual) address space size is assumed to be PLAT_PHY_ADDR_SPACE_SIZE
(resp. PLAT_VIRT_ADDR_SPACE_SIZE).

This patch removes them and reads the relevant address space size
from the translation context itself instead. This information is now
passed in argument to the enable_mmu_arch() function, which needs it
to configure the TCR_ELx.T0SZ field (in AArch64) or the TTBCR.T0SZ
field (in AArch32) appropriately.

Change-Id: I20b0e68b03a143e998695d42911d9954328a06aa
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

This patch refactors both the AArch32 and AArch64 versions of the
function enable_mmu_arch().

In both versions, the code now computes the VMSA-related system
registers upfront then program them in one go (rather than interleaving
the 2).

In the AArch64 version, this allows to reduce the amount of code
generated by the C preprocessor and limits it to the actual differences
between EL1 and EL3.

In the AArch32 version, this patch also removes the function
enable_mmu_internal_secure() and moves its code directly inside
enable_mmu_arch(), as it was its only caller.

Change-Id: I35c09b6db4404916cbb2e2fd3fda2ad59f935954
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

In both the AArch32 and AArch64 versions, this function used to check
the sanity of the PLAT_PHY_ADDR_SPACE_SIZE in regard to the
architectural maximum value. Instead, export the
xlat_arch_get_max_supported_pa() function and move the debug
assertion in AArch-agnostic code.

The AArch64 used to also precalculate the TCR.PS field value, based
on the size of the physical address space. This is now done directly
by enable_mmu_arch(), which now receives the physical address space size
in argument.

Change-Id: Ie77ea92eb06db586f28784fdb479c6e27dd1acc1
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

TLBI instructions for monitor mode won't have the desired effect under
specific circumstances in Cortex-A57 r0p0. The workaround is to
execute DSB and TLBI twice each time.

Even though this errata is only needed in r0p0, the current errata
framework is not prepared to apply run-time workarounds. The current one
is always applied if compiled in, regardless of the CPU or its revision.

The `DSB` instruction used when initializing the translation tables has
been changed to `DSB ISH` as an optimization and to be consistent with
the barriers used for the workaround.

NOTE: This workaround is present in AArch64 TF and already enabled by
default on Juno.

Change-Id: I10b0baa304ed64b13b7b26ea766e61461e759dfa
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/



Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Translation regimes that only support one virtual address space (such as
the ones for EL2 and EL3) can flag memory regions as execute-never by
setting to 1 the XN bit in the Upper Attributes field in the translation
tables descriptors. Translation regimes that support two different
virtual address spaces (such as the one shared by EL1 and EL0) use bits
PXN and UXN instead.

The Trusted Firmware runs at EL3 and EL1, it has to handle translation
tables of both translation regimes, but the previous code handled both
regimes the same way, as if both had only 1 VA range.

When trying to set a descriptor as execute-never it would set the XN
bit correctly in EL3, but it would set the XN bit in EL1 as well. XN is
at the same bit position as UXN, which means that EL0 was being
prevented from executing code at this region, not EL1 as the code
intended. Therefore, the PXN bit was unset to 0 all the time. The result
is that, in AArch64 mode, read-only data sections of BL2 weren't
protected from being executed.

This patch adds support of translation regimes with two virtual address
spaces to both versions of the translation tables library, fixing the
execute-never permissions for translation tables in EL1.

The library currently does not support initializing translation tables
for EL0 software, therefore it does not set/unset the UXN bit. If EL1
software needs to initialize translation tables for EL0 software, it
should use a different library instead.

Change-Id: If27588f9820ff42988851d90dc92801c8ecbe0c9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Many asserts depend on code that is conditionally compiled based on the
DEBUG define. This patch modifies the conditional inclusion of such code
so that it is based on the ENABLE_ASSERTIONS build option.

Change-Id: I6406674788aa7e1ad7c23d86ce94482ad3c382bd
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This patch adds an additional flag `XLAT_TABLE_NC` which marks the
translation tables as Non-cacheable for MMU accesses.

Change-Id: I7c28ab87f0ce67da237fadc3627beb6792860fd4
Signed-off-by: Summer Qin <summer.qin@arm.com>

Added APIs to add and remove regions to the translation tables
dynamically while the MMU is enabled. Only static regions are allowed
to overlap other static ones (for backwards compatibility).

A new private attribute (MT_DYNAMIC / MT_STATIC) has been added to
flag each region as such.

The dynamic mapping functionality can be enabled or disabled when
compiling by setting the build option PLAT_XLAT_TABLES_DYNAMIC to 1
or 0. This can be done per-image.

TLB maintenance code during dynamic table mapping and unmapping has
also been added.

Fixes ARM-software/tf-issues#310

Change-Id: I19e8992005c4292297a382824394490c5387aa3b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

The folder lib/xlat_tables_v2 has been created to store a new version
of the translation tables library for further modifications in patches
to follow. At the moment it only contains a basic implementation that
supports static regions.

This library allows different translation tables to be modified by
using different 'contexts'. For now, the implementation defaults to
the translation tables used by the current image, but it is possible
to modify other tables than the ones in use.

Added a new API to print debug information for the current state of
the translation tables, rather than printing the information while
the tables are being created. This allows subsequent debug printing
of the xlat tables after they have been changed, which will be useful
when dynamic regions are implemented in a patch to follow.

The common definitions stored in `xlat_tables.h` header have been moved
to a new file common to both versions, `xlat_tables_defs.h`.

All headers related to the translation tables library have been moved to
a the subfolder `xlat_tables`.

Change-Id: Ia55962c33e0b781831d43a548e505206dffc5ea9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Added the definitions `PLAT_PHY_ADDR_SPACE_SIZE` and
`PLAT_VIRT_ADDR_SPACE_SIZE` which specify respectively the physical
and virtual address space size a platform can use.

`ADDR_SPACE_SIZE` is now deprecated. To maintain compatibility, if any
of the previous defines aren't present, the value of `ADDR_SPACE_SIZE`
will be used instead.

For AArch64, register ID_AA64MMFR0_EL1 is checked to calculate the
max PA supported by the hardware and to verify that the previously
mentioned definition is valid. For AArch32, a 40 bit physical
address space is considered.

Added asserts to check for overflows.

Porting guide updated.

Change-Id: Ie8ce1da5967993f0c94dbd4eb9841fc03d5ef8d6
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Instead of hardcoding a level 1 table as the base translation level
table, let the code decide which level is the most appropriate given
the virtual address space size.

As the table granularity is 4 KB, this allows the code to select
level 0, 1 or 2 as base level for AArch64. This way, instead of
limiting the virtual address space width to 39-31 bits, widths of
48-25 bit can be used.

For AArch32, this change allows the code to select level 1 or 2
as the base translation level table and use virtual address space
width of 32-25 bits.

Also removed some unused definitions related to translation tables.

Fixes ARM-software/tf-issues#362

Change-Id: Ie3bb5d6d1a4730a26700b09827c79f37ca3cdb65

This patch adds translation library supports for AArch32 platforms.
The library only supports long descriptor formats for AArch32.
The `enable_mmu_secure()` enables the MMU for secure world with
`TTBR0` pointing to the populated translation tables.

Change-Id: I061345b1779391d098e35e7fe0c76e3ebf850e08