GitLab

This patch adds a driver for ARM GICv3 systems that need to run software
stacks where affinity routing is enabled across all privileged exception
levels for both security states. This driver is a partial implementation
of the ARM Generic Interrupt Controller Architecture Specification, GIC
architecture version 3.0 and version 4.0 (ARM IHI 0069A). The driver does
not cater for legacy support of interrupts and asymmetric configurations.

The existing GIC driver has been preserved unchanged. The common code for
GICv2 and GICv3 systems has been refactored into a new file,
`drivers/arm/gic/common/gic_common.c`. The corresponding header is in
`include/drivers/arm/gic_common.h`.

The driver interface is implemented in `drivers/arm/gic/v3/gicv3_main.c`.
The corresponding header is in `include/drivers/arm/gicv3.h`. Helper
functions are implemented in `drivers/arm/gic/v3/arm_gicv3_helpers.c`
and are accessible through the `drivers/arm/gic/v3/gicv3_private.h`
header.

Change-Id: I8c3c834a1d049d05b776b4dcb76b18ccb927444a

On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).

This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:

1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
   initialised. This ensures that any stale cache lines at any level of cache
   are removed.

2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
   visible to secondary CPUs using a cache clean operation by MVA.

3. Cache maintenance by set/way operations are only used prior to power down.

NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.

Fixes ARM-software/tf-issues#205

Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a

The required platform constant PLATFORM_CACHE_LINE_SIZE is
unnecessary since CACHE_WRITEBACK_GRANULE effectively provides the
same information. CACHE_WRITEBACK_GRANULE is preferred since this
is an architecturally defined term and allows comparison with the
corresponding hardware register value.

Replace all usage of PLATFORM_CACHE_LINE_SIZE with
CACHE_WRITEBACK_GRANULE.

Also, add a runtime assert in BL1 to check that the provided
CACHE_WRITEBACK_GRANULE matches the value provided in CTR_EL0.

Change-Id: If87286be78068424217b9f3689be358356500dcd

This patch adds functionality to translate virtual addresses from
secure or non-secure worlds. This functionality helps Trusted Apps
to share virtual addresses directly and allows the NS world to
pass virtual addresses to TLK directly.

Change-Id: I77b0892963e0e839c448b5d0532920fb7e54dc8e
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

This patch adds support to return SUCCESS if a pending interrupt is
detected during a CPU_SUSPEND call to a power down state. The check
is performed as late as possible without losing the ability to return
to the caller. This reduces the overhead incurred by a CPU in
undergoing a complete power cycle when a wakeup interrupt is already
pending.

Fixes ARM-Software/tf-issues#102

Change-Id: I1aff04a74b704a2f529734428030d1d10750fd4b

This patch adds helper macros for barrier operations that specify
the type of barrier (dmb, dsb) and the shareability domain (system,
inner-shareable) it affects.

Change-Id: I4bf95103e79da212c4fbdbc13d91ad8ac385d9f5

Calls to system register read accessors functions may be optimised
out by the compiler if called twice in a row for the same register.
This is because the compiler is not aware that the result from
the instruction may be modified by external agents. Therefore, if
nothing modifies the register between the 2 reads as far as the
compiler knows then it might consider that it is useless to read
it twice and emit only 1 call.

This behaviour is faulty for registers that may not have the same
value if read twice in succession. E.g.: counters, timer
control/countdown registers, GICv3 interrupt status registers and
so on.

The same problem happens for calls to system register write
accessors functions. The compiler might optimise out some calls
if it considers that it will produce the same result. Again, this
behaviour is faulty for cases where intermediate writes to these
registers make a difference in the system.

This patch fixes the problem by making these assembly register
accesses volatile.

Fixes ARM-software/tf-issues#273

Change-Id: I33903bc4cc4eea8a8d87bc2c757909fbb0138925

This patch resets the value of CNTVOFF_EL2 before exit to EL1 on
warm boot. This needs to be done if only the Trusted Firmware exits
to EL1 instead of EL2, otherwise the hypervisor would be responsible
for this.

Fixes ARM-software/tf-issues#240

Change-Id: I79d54831356cf3215bcf1f251c373bd8f89db0e0

This patch adds CPU core and cluster power down sequences to the CPU specific
operations framework introduced in a earlier patch. Cortex-A53, Cortex-A57 and
generic AEM sequences have been added. The latter is suitable for the
Foundation and Base AEM FVPs. A pointer to each CPU's operations structure is
saved in the per-cpu data so that it can be easily accessed during power down
seqeunces.

An optional platform API has been introduced to allow a platform to disable the
Accelerator Coherency Port (ACP) during a cluster power down sequence. The weak
definition of this function (plat_disable_acp()) does not take any action. It
should be overriden with a strong definition if the ACP is present on a
platform.

Change-Id: I8d09bd40d2f528a28d2d3f19b77101178778685d

Exclude stdlib files because they do not follow kernel code style.

Fixes ARM-software/tf-issues#73

Change-Id: I4cfafa38ab436f5ab22c277cb38f884346a267ab

Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
*  The first uses entry_point_info to initialize the relevant
   cpu_context for first entry into a lower exception level on a CPU
*  The second populates the EL1 and EL2 system registers as needed
   from the cpu_context to ensure correct entry into the lower EL

This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).

In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.

Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f

Replace the current out-of-line assembler implementations of
the system register and system instruction operations with
inline assembler.

This enables better compiler optimisation and code generation
when accessing system registers.

Fixes ARM-software/tf-issues#91

Change-Id: I149af3a94e1e5e5140a3e44b9abfc37ba2324476

Function declarations implicitly have external linkage so do not
need the extern keyword.

Change-Id: Ia0549786796d8bf5956487e8996450a0b3d79f32

This patch adds support in the TSP to program the secure physical
generic timer to generate a EL-1 interrupt every half second. It also
adds support for maintaining the timer state across power management
operations. The TSPD ensures that S-EL1 can access the timer by
programming the SCR_EL3.ST bit.

This patch does not actually enable the timer. This will be done in a
subsequent patch once the complete framework for handling S-EL1
interrupts is in place.

Change-Id: I1b3985cfb50262f60824be3a51c6314ce90571bc

This patch implements the register reporting when unhandled exceptions are
taken in BL3-1. Unhandled exceptions will result in a dump of registers
to the console, before halting execution by that CPU. The Crash Stack,
previously called the Exception Stack, is used for this activity.
This stack is used to preserve the CPU context and runtime stack
contents for debugging and analysis.

This also introduces the per_cpu_ptr_cache, referenced by tpidr_el3,
to provide easy access to some of BL3-1 per-cpu data structures.
Initially, this is used to provide a pointer to the Crash stack.

panic() now prints the the error file and line number in Debug mode
and prints the PC value in release mode.

The Exception Stack is renamed to Crash Stack with this patch.
The original intention of exception stack is no longer valid
since we intend to support several valid exceptions like IRQ
and FIQ in the trusted firmware context. This stack is now
utilized for dumping and reporting the system state when a
crash happens and hence the rename.

Fixes ARM-software/tf-issues#79 Improve reporting of unhandled exception

Change-Id: I260791dc05536b78547412d147193cdccae7811a

The goal of these macros is to improve code readability by providing
a concise way to check whether we are running in the expected
exception level.

Change-Id: If9aebadfb6299a5196e9a582b442f0971d9909b1

There are a small number of non-EL specific helper functions
which are no longer used, and also some unusable helper
functions for non-existant registers.

This change removes all of these functions.

Change-Id: Idd656cef3b59cf5c46fe2be4029d72288b649c24

disable_mmu() cannot work as a C function as there is no control
over data accesses generated by the compiler between disabling and
cleaning the data cache. This results in reading stale data from
main memory.

As assembler version is provided for EL3, and a variant that also
disables the instruction cache which is now used by the BL1
exception handling function.

Fixes ARM-software/tf-issues#147

Change-Id: I0cf394d2579a125a23c2f2989c2e92ace6ddb1a6

Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.

Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.

Fixes ARM-software/tf-issues#31

Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e

Move almost all system include files to a logical sub-directory
under ./include. The only remaining system include directories
not under ./include are specific to the platform. Move the
corresponding source files to match the include directory
structure.

Also remove pm.h as it is no longer used.

Change-Id: Ie5ea6368ec5fad459f3e8a802ad129135527f0b3

Each ARM Trusted Firmware image should know in which EL it is running
and it should use the corresponding register directly instead of reading
currentEL and knowing which asm register to read/write

Change-Id: Ief35630190b6f07c8fbb7ba6cb20db308f002945

This patch factors out the ARM FVP specific code to create MMU
translation tables so that it is possible for a boot loader stage to
create a different set of tables instead of using the default ones.
The default translation tables are created with the assumption that
the calling boot loader stage executes out of secure SRAM. This might
not be true for the BL3_2 stage in the future.

A boot loader stage can define the `fill_xlation_tables()` function as
per its requirements. It returns a reference to the level 1
translation table which is used by the common platform code to setup
the TTBR_EL3.

This patch is a temporary solution before a larger rework of
translation table creation logic is introduced.

Change-Id: I09a075d5da16822ee32a411a9dbe284718fb4ff6

This patch reworks BL2 to BL3-1 hand over interface by introducing a
composite structure (bl31_args) that holds the superset of information
that needs to be passed from BL2 to BL3-1.

  - The extents of secure memory available to BL3-1
  - The extents of memory available to BL3-2 (not yet implemented) and
    BL3-3
  - Information to execute BL3-2 (not yet implemented) and BL3-3 images

This patch also introduces a new platform API (bl2_get_bl31_args_ptr)
that needs to be implemented by the platform code to export reference to
bl31_args structure which has been allocated in platform-defined memory.

The platform will initialize the extents of memory available to BL3-3
during early platform setup in bl31_args structure. This obviates the
need for bl2_get_ns_mem_layout platform API.

BL2 calls the bl2_get_bl31_args_ptr function to get a reference to
bl31_args structure. It uses the 'bl33_meminfo' field of this structure
to load the BL3-3 image. It sets the entry point information for the
BL3-3 image in the 'bl33_image_info' field of this structure. The
reference to this structure is passed to the BL3-1 image.

Also fixes issue ARM-software/tf-issues#25

Change-Id: Ic36426196dd5ebf89e60ff42643bed01b3500517

Change-Id: Ic7fb61aabae1d515b9e6baf3dd003807ff42da60

This patch removes the duplicate declaration of psci_cpu_on in psci.h
and moves the constants for the system level implementation of the
generic timer from arch_helpers.h to arch.h. All other architectural
constants are defined in arch.h so there is no need to add them to
arch_helpers.h

Change-Id: Ia8ad3f91854f7e57fce31873773eede55c384ff1

- Add instructions for contributing to ARM Trusted Firmware.

- Update copyright text in all files to acknowledge contributors.

Change-Id: I9311aac81b00c6c167d2f8c889aea403b84450e5

Also, don't invalidate the TLBs in disable_mmu() function, it's better
to do it in enable_mmu() function just before actually enabling the
MMU.

Change-Id: Ib32d6660019b0b2c17254156aad4be67ab4970e1

Any asynchronous exception caused by the firmware should be handled
in the firmware itself.  For this reason, unmask SError exceptions
(and Debug ones as well) on all boot paths.  Also route external
abort and SError interrupts to EL3, otherwise they will target EL1.

Change-Id: I9c191d2d0dcfef85f265641c8460dfbb4d112092