GitLab

This patch introduces the ability of the xlat tables library to manage
EL0 and EL1 mappings from a higher exception level.

Attributes MT_USER and MT_PRIVILEGED have been added to allow the user
specify the target EL in the translation regime EL1&0.

REGISTER_XLAT_CONTEXT2 macro is introduced to allow creating a
xlat_ctx_t that targets a given translation regime (EL1&0 or EL3).

A new member is added to xlat_ctx_t to represent the translation regime
the xlat_ctx_t manages. The execute_never mask member is removed as it
is computed from existing information.

Change-Id: I95e14abc3371d7a6d6a358cc54c688aa9975c110
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Introduce a variant of the TLB invalidation helper function that
allows the targeted translation regime to be specified, rather than
defaulting to the current one.

This new function is useful in the context of EL3 software managing
translation tables for the S-EL1&0 translation regime, as then it
might need to invalidate S-EL1&0 TLB entries rather than EL3 ones.

Define a new enumeration to be able to represent translation regimes in
the xlat tables library.

Change-Id: Ibe4438dbea2d7a6e7470bfb68ff805d8bf6b07e5
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Douglas Raillard <douglas.raillard@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

TLB invalidation functions used to be conditionally compiled in.
They were enabled only when using the dynamic mapping feature.
because only then would we need to modify page tables on the fly.

Actually there are other use cases where invalidating TLBs is required.
When changing memory attributes in existing translation descriptors for
example. These other use cases do not necessarily depend on the dynamic
mapping feature.

This patch removes this dependency and always compile TLB invalidation
functions in. If they're not used, they will be removed from the binary
at link-time anyway so there's no consequence on the memory footprint
if these functions are not called.

Change-Id: I1c33764ae900eb00073ee23b7d0d53d4efa4dd21
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

The current implementation of the memory mapping API favours mapping
memory regions using the biggest possible block size in order to
reduce the number of translation tables needed.

In some cases, this behaviour might not be desirable. When translation
tables are edited at run-time, coarse-grain mappings like that might
need splitting into finer-grain tables. This operation has a
performance cost.

The MAP_REGION2() macro allows to specify the granularity of
translation tables used for the initial mapping of a memory region.
This might increase performance for memory regions that are likely to
be edited in the future, at the expense of a potentially increased
memory footprint.

The Translation Tables Library Design Guide has been updated to
explain the use case for this macro. Also added a few intermediate
titles to make the guide easier to digest.

Change-Id: I04de9302e0ee3d326b8877043a9f638766b81b7b
Co-authored-by: Sandrine Bailleux <sandrine.bailleux@arm.com>
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

In the S-EL1&0 translation regime we aren't using the higher VA range,
whose translation table base address is held in TTBR1_EL1. The bit
TCR_EL1.EPD1 can be used to disable translations using TTBR1_EL1, but
the code wasn't setting it to 1. Additionally, other fields in TCR1_EL1
associated with the higher VA range (TBI1, TG1, SH1, ORGN1, IRGN1 and
A1) weren't set correctly as they were left as 0. In particular, 0 is a
reserved value for TG1. Also, TBBR1_EL1 was not explicitly set and its
reset value is UNKNOWN.

Therefore memory accesses to the higher VA range would result in
unpredictable behaviour as a translation table walk would be attempted
using an UNKNOWN value in TTBR1_EL1.

On the FVP and Juno platforms accessing the higher VA range resulted in
a translation fault, but this may not always be the case on all
platforms.

This patch sets the bit TCR_EL1.EPD1 to 1 so that any kind of
unpredictable behaviour is prevented.

This bug only affects the AArch64 version of the code, the AArch32
version sets this bit to 1 as expected.

Change-Id: I481c000deda5bc33a475631301767b9e0474a303
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Use a mmap_region_t as parameter instead of getting a parameter for each
structure member. This reduces the scope of changes when adding members
to mmap_region_t.

Also align on the convention of using mm_cursor as a variable name for
the currently inspected region when iterating on the region array.

Change-Id: If40bc4351b56c64b214e60dda27276d11ce9dbb3
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

mmap_add_region_ctx and mmap_add_dynamic_region_ctx are clearing members
that they are not aware of by copying each member one by one. Replace
this by structure assignment.

Change-Id: I7c70cb408c8a8eb551402a5d8d956c1fb7f32b55
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Use the MAP_REGION to build the mmap_region_t argument in wrappers like
mmap_add_region(). Evolution of the mmap_region_t might require adding
new members with a non-zero default value. Users of MAP_REGION are
protected against such evolution. This commit also protects users of
mmap_add_region() and mmap_add_dynamic_region() functions against these
evolutions.

Also make the MAP_REGION macro implementation more explicit and make it
a mmap_region_t compound literal to make it useable as a function
parameter on its own and to prevent using it in initialization of
variables of different type.

Change-Id: I7bfc4689f6dd4dd23c895b65f628d8ee991fc161
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

This patch enables the CnP (Common not Private) bit for secure page
tables so that multiple PEs in the same Inner Shareable domain can use
the same translation table entries for a given stage of translation in
a particular translation regime. This only takes effect when ARM
Trusted Firmware is built with ARM_ARCH_MINOR >= 2.

ARM Trusted Firmware Design has been updated to include a description
of this feature usage.

Change-Id: I698305f047400119aa1900d34c65368022e410b8
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>

This patch changes the sign of the loop variable used in
xlat_tables_print(). It needs to be unsigned because it is compared
against another unsigned int.

Change-Id: I2b3cee7990dd75e8ebd2701de3860ead7cad8dc8
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

Fix the type length and signedness of some of the constants and
variables used in the translation table library.

This patch supersedes Pull Request #1018:
https://github.com/ARM-software/arm-trusted-firmware/pull/1018



Change-Id: Ibd45faf7a4fb428a0bf71c752551d35800212fb2
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

Previous patches have made it possible to specify the physical and
virtual address spaces sizes for each translation context. However,
there are still some places in the code where the physical (resp.
virtual) address space size is assumed to be PLAT_PHY_ADDR_SPACE_SIZE
(resp. PLAT_VIRT_ADDR_SPACE_SIZE).

This patch removes them and reads the relevant address space size
from the translation context itself instead. This information is now
passed in argument to the enable_mmu_arch() function, which needs it
to configure the TCR_ELx.T0SZ field (in AArch64) or the TTBCR.T0SZ
field (in AArch32) appropriately.

Change-Id: I20b0e68b03a143e998695d42911d9954328a06aa
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

This patch refactors both the AArch32 and AArch64 versions of the
function enable_mmu_arch().

In both versions, the code now computes the VMSA-related system
registers upfront then program them in one go (rather than interleaving
the 2).

In the AArch64 version, this allows to reduce the amount of code
generated by the C preprocessor and limits it to the actual differences
between EL1 and EL3.

In the AArch32 version, this patch also removes the function
enable_mmu_internal_secure() and moves its code directly inside
enable_mmu_arch(), as it was its only caller.

Change-Id: I35c09b6db4404916cbb2e2fd3fda2ad59f935954
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

In both the AArch32 and AArch64 versions, this function used to check
the sanity of the PLAT_PHY_ADDR_SPACE_SIZE in regard to the
architectural maximum value. Instead, export the
xlat_arch_get_max_supported_pa() function and move the debug
assertion in AArch-agnostic code.

The AArch64 used to also precalculate the TCR.PS field value, based
on the size of the physical address space. This is now done directly
by enable_mmu_arch(), which now receives the physical address space size
in argument.

Change-Id: Ie77ea92eb06db586f28784fdb479c6e27dd1acc1
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

In a previous patch, the xlat_ctx_t type has been made public.
This patch now makes the *_ctx() APIs public.

Each API now has a *_ctx() variant. Most of them were already implemented
and this patch just makes them public. However, some of them were missing
so this patch introduces them.

Now that all these APIs are public, there's no good reason for splitting
them accross 2 files (xlat_tables_internal.c and xlat_tables_common.c).
Therefore, this patch moves all code into xlat_tables_internal.c and
removes xlat_tables_common.c. It removes it from the library's makefile
as well.

This last change introduces a compatibility break for platform ports
that specifically include the xlat_tables_common.c file instead of
including the library's Makefile. The UniPhier platform makefile has
been updated to now omit this file from the list of source files.

The prototype of mmap_add_region_ctx() has been slightly changed. The
mmap_region_t passed in argument needs to be constant because it gets
called from map_add(), which receives a constant region. The former
implementation of mmap_add() used to cast the const qualifier away,
which is not a good practice.

Also remove init_xlation_table(), which was a sub-function of
init_xlat_tables(). Now there's just init_xlat_tables() (and
init_xlat_tables_ctx()). Both names were too similar, which was
confusing. Besides, now that all the code is in a single file,
it's no longer needed to have 2 functions for that.

Change-Id: I4ed88c68e44561c3902fbebb89cb197279c5293b
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

At the moment, the translation context type (xlat_ctx_t) is a private
type reserved for the internal usage of the translation table library.
All exported APIs (implemented in xlat_tables_common.c) are wrappers
over the internal implementations that use such a translation context.

These wrappers unconditionally pass the current translation context
representing the memory mappings of the executing BL image. This means
that the caller has no control over which translation context the
library functions act on.

As a first step to make this code more flexible, this patch exports
the 'xlat_ctx_t' type. Note that, although the declaration of this type
is now public, its definition stays private. A macro is introduced to
statically allocate and initialize such a translation context.

The library now internally uses this macro to allocate the default
translation context for the running BL image.

Change-Id: Icece1cde4813fac19452c782b682c758142b1489
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

Move the header files that provide translation tables architectural
definitions from the library v2 source files to the library include
directory. This allows to share these definitions between both
versions (v1 and v2) of the library.

Create a new header file that includes the AArch32 or AArch64
definitions based on the AARCH32 build flag, so that the library user
doesn't have to worry about handling it on their side.

Also repurpose some of the definitions the header files provide to
concentrate on the things that differ between AArch32 and AArch64.
As a result they now contain the following information:
 - the first table level that allows block descriptors;
 - the architectural limits of the virtual address space;
 - the initial lookup level to cover the entire address space.

Additionally, move the XLAT_TABLE_LEVEL_MIN macro from
xlat_tables_defs.h to the AArch32/AArch64 architectural definitions.

This new organisation eliminates duplicated information in the AArch32
and AArch64 versions. It also decouples these architectural files from
any platform-specific information. Previously, they were dependent on
the address space size, which is platform-specific.

Finally, for the v2 of the library, move the compatibility code for
ADDR_SPACE_SIZE into a C file as it is not needed outside of this
file. For v1, this code hasn't been changed and stays in a header
file because it's needed by several files.

Change-Id: If746c684acd80eebf918abd3ab6e8481d004ac68
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

This patch adds some debug prints to display some statistics about page
tables usage. They are printed only if the LOG_LEVEL is at least 50
(i.e. VERBOSE).

Sample output for BL1:

VERBOSE:    Translation tables state:
VERBOSE:      Max allowed PA:  0xffffffff
VERBOSE:      Max allowed VA:  0xffffffff
VERBOSE:      Max mapped PA:   0x7fffffff
VERBOSE:      Max mapped VA:   0x7fffffff
VERBOSE:      Initial lookup level: 1
VERBOSE:      Entries @initial lookup level: 4
VERBOSE:      Used 4 sub-tables out of 5 (spare: 1)

Change-Id: If38956902e9616cdcd6065ecd140fe21482597ea
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

TLBI instructions for monitor mode won't have the desired effect under
specific circumstances in Cortex-A57 r0p0. The workaround is to
execute DSB and TLBI twice each time.

Even though this errata is only needed in r0p0, the current errata
framework is not prepared to apply run-time workarounds. The current one
is always applied if compiled in, regardless of the CPU or its revision.

The `DSB` instruction used when initializing the translation tables has
been changed to `DSB ISH` as an optimization and to be consistent with
the barriers used for the workaround.

NOTE: This workaround is present in AArch64 TF and already enabled by
default on Juno.

Change-Id: I10b0baa304ed64b13b7b26ea766e61461e759dfa
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

This patch changes input param level in xlat_tables_print_internal() to
an unsigned int to fix the signed/unsigned comparison warnings. The
compiler complains about these warnings, thus halting the build flow
for Tegra platforms.

Change-Id: Ieccc262a63daca7a26ca6a14d81466397af8b89f
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Signed-off-by: David Cunado <david.cunado@arm.com>

This patch enables the 'sign-compare' flag, to enable warning/errors
for comparisons between signed/unsigned variables. The warning has
been enabled for all the Tegra platforms, to start with.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

xlat_tables_arch.h uses the platform macro `PLAT_VIRT_ADDR_SPACE_SIZE`.
This macro is defined in xlat_tables_private.h only if the platform
still uses the deprecated `ADDR_SPACE_SIZE`.

Change-Id: I1c3b12ebd96bdfe9bf94b26d440c03bc0f8c0b24
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

- Fix some comments.
- Remove duplicated definition.
- Make xlat_arch_get_max_supported_pa() private in aarch64.

Change-Id: I629237209cfb2ce7b0c4bd539d63dd81d45b2edd
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.

NOTE: Files that have been imported by FreeBSD have not been modified.

[0]: https://spdx.org/



Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

Translation regimes that only support one virtual address space (such as
the ones for EL2 and EL3) can flag memory regions as execute-never by
setting to 1 the XN bit in the Upper Attributes field in the translation
tables descriptors. Translation regimes that support two different
virtual address spaces (such as the one shared by EL1 and EL0) use bits
PXN and UXN instead.

The Trusted Firmware runs at EL3 and EL1, it has to handle translation
tables of both translation regimes, but the previous code handled both
regimes the same way, as if both had only 1 VA range.

When trying to set a descriptor as execute-never it would set the XN
bit correctly in EL3, but it would set the XN bit in EL1 as well. XN is
at the same bit position as UXN, which means that EL0 was being
prevented from executing code at this region, not EL1 as the code
intended. Therefore, the PXN bit was unset to 0 all the time. The result
is that, in AArch64 mode, read-only data sections of BL2 weren't
protected from being executed.

This patch adds support of translation regimes with two virtual address
spaces to both versions of the translation tables library, fixing the
execute-never permissions for translation tables in EL1.

The library currently does not support initializing translation tables
for EL0 software, therefore it does not set/unset the UXN bit. If EL1
software needs to initialize translation tables for EL0 software, it
should use a different library instead.

Change-Id: If27588f9820ff42988851d90dc92801c8ecbe0c9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

utils.h is included in various header files for the defines in it.
Some of the other header files only contain defines.  This allows the
header files to be shared between host and target builds for shared defines.

Recently types.h has been included in utils.h as well as some function
prototypes.

Because of the inclusion of types.h conflicts exist building host tools
abd these header files now.  To solve this problem,
move the defines to utils_def.h and have this included by utils.h and
change header files to only include utils_def.h and not pick up the new
types.h being introduced.

Fixes ARM-software/tf-issues#461
Signed-off-by: Scott Branden <scott.branden@broadcom.com>

Remove utils_def.h from utils.h

This patch removes utils_def.h from utils.h as it is not required.
And also makes a minor change to ensure Juno platform compiles.

Change-Id: I10cf1fb51e44a8fa6dcec02980354eb9ecc9fa29

This patch modifies both versions of the translation table library
to use the mmap_attr_t type consistently wherever it is manipulating
MT_* attributes variables. It used to use mmap_attr_t or plain integer
types interchangeably, which compiles fine because an enumeration type
can be silently converted to an integer, but which is semantically
incorrect.

This patch removes this assumption by using the abstract type
'mmap_attr_t' all the time.

Change-Id: Id1f099025d2cb962b275bb7e39ad2c4dbb4e366c
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

Many asserts depend on code that is conditionally compiled based on the
DEBUG define. This patch modifies the conditional inclusion of such code
so that it is based on the ENABLE_ASSERTIONS build option.

Change-Id: I6406674788aa7e1ad7c23d86ce94482ad3c382bd
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

This patch adds an additional flag `XLAT_TABLE_NC` which marks the
translation tables as Non-cacheable for MMU accesses.

Change-Id: I7c28ab87f0ce67da237fadc3627beb6792860fd4
Signed-off-by: Summer Qin <summer.qin@arm.com>

TLBI instructions for EL3 won't have the desired effect under specific
circumstances in Cortex-A57 r0p0. The workaround is to execute DSB and
TLBI twice each time.

Even though this errata is only needed in r0p0, the current errata
framework is not prepared to apply run-time workarounds. The current one
is always applied if compiled in, regardless of the CPU or its revision.

This errata has been enabled for Juno.

The `DSB` instruction used when initializing the translation tables has
been changed to `DSB ISH` as an optimization and to be consistent with
the barriers used for the workaround.

Change-Id: Ifc1d70b79cb5e0d87e90d88d376a59385667d338
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Added APIs to add and remove regions to the translation tables
dynamically while the MMU is enabled. Only static regions are allowed
to overlap other static ones (for backwards compatibility).

A new private attribute (MT_DYNAMIC / MT_STATIC) has been added to
flag each region as such.

The dynamic mapping functionality can be enabled or disabled when
compiling by setting the build option PLAT_XLAT_TABLES_DYNAMIC to 1
or 0. This can be done per-image.

TLB maintenance code during dynamic table mapping and unmapping has
also been added.

Fixes ARM-software/tf-issues#310

Change-Id: I19e8992005c4292297a382824394490c5387aa3b
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

The printed output has been improved in two ways:

- Whenever multiple invalid descriptors are found, only the first one
  is printed, and a line is added to inform about how many descriptors
  have been omitted.

- At the beginning of each line there is an indication of the table
  level the entry belongs to. Example of the new output:
  `[LV3] VA:0x1000 PA:0x1000 size:0x1000 MEM-RO-S-EXEC`

Change-Id: Ib6f1cd8dbd449452f09258f4108241eb11f8d445
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

The folder lib/xlat_tables_v2 has been created to store a new version
of the translation tables library for further modifications in patches
to follow. At the moment it only contains a basic implementation that
supports static regions.

This library allows different translation tables to be modified by
using different 'contexts'. For now, the implementation defaults to
the translation tables used by the current image, but it is possible
to modify other tables than the ones in use.

Added a new API to print debug information for the current state of
the translation tables, rather than printing the information while
the tables are being created. This allows subsequent debug printing
of the xlat tables after they have been changed, which will be useful
when dynamic regions are implemented in a patch to follow.

The common definitions stored in `xlat_tables.h` header have been moved
to a new file common to both versions, `xlat_tables_defs.h`.

All headers related to the translation tables library have been moved to
a the subfolder `xlat_tables`.

Change-Id: Ia55962c33e0b781831d43a548e505206dffc5ea9
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>