GitLab

This patch checks if standard SMC calls, meant for TLK, are issued
only on the boot CPU. TLK is UP Trusted OS stack and so we need this
check to avoid the NS world calling into TLK from any other CPU.

The previous check tied TLK to CPU0, but the boot CPU can be other
than CPU0 in some scenarios.

Change-Id: I75eaafa32471ce19e9920433c2f97b6b5fc02d86
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

This patch passes the boot parameters, provided by the previous
bootloader, to the Trusted OS via X0, X1 and X2.

Original change by: Wayne Lin <wlin@nvidia.com>

Change-Id: I2039612a8a8226158babfd505ce8c31c4212319c
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

In multi-guest trusty environment, all guest's SMCs will be
forwarded to Trusty. This change only allows 1 guest's SMC
to be forwarded at a time and returns 'busy' status to all
other requests.

Change-Id: I2144467d11e3680e28ec816adeec2766bca114d4
Signed-off-by: Anthony Zhou <anzhou@nvidia.com>
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

According to the ARM DEN0028A spec, hypervisor ID(VMID) should be stored
in x7 (or w7). This patch gets this value from the context and passes it
to Trusty. In order to do so, introduce new macros to pass five to eight
parameters to the Trusted OS.

Change-Id: I101cf45d0712e1e880466b2274f9a48af755c9fa
Signed-off-by: Anthony Zhou <anzhou@nvidia.com>
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

This patch uses the OEN_TAP_START aperture for all the standard
calls being passed to Trusty.

Change-Id: Id78d01c7f48e4f54855600d7c789ffbfb898c541
Signed-off-by: Amith <aramachan@nvidia.com>
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

ABORT SMC used to return to the previously executing world, which
happened to be S-EL1 as it calls a TSP handler using synchronous entry
into the TSP.

Now properly save and restore the non-secure context (including system
registers) and return to non-secure world as it should.

fixes ARM-Software/tf-issues#453

Change-Id: Ie40c79ca2636ab8b6b2ab3106e8f49e0f9117f5f
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Replace all use of memset by zeromem when zeroing moderately-sized
structure by applying the following transformation:
memset(x, 0, sizeof(x)) => zeromem(x, sizeof(x))

As the Trusted Firmware is compiled with -ffreestanding, it forbids the
compiler from using __builtin_memset and forces it to generate calls to
the slow memset implementation. Zeromem is a near drop in replacement
for this use case, with a more efficient implementation on both AArch32
and AArch64.

Change-Id: Ia7f3a90e888b96d056881be09f0b4d65b41aa79e
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

With GCC 6.2 compiler, more C undefined behaviour is being flagged as
warnings, which result in build errors in ARM TF build.

The specific issue that this patch resolves is the use of (1 << 31),
which is predominantly used in case statements, where 1 is represented
as a signed int. When shifted to msb the behaviour is undefined.

The resolution is to specify 1 as an unsigned int using a convenience
macro ULL(). A duplicate macro MAKE_ULL() is replaced.

Fixes ARM-software/tf-issues#438

Change-Id: I08e3053bbcf4c022ee2be33a75bd0056da4073e1
Signed-off-by: David Cunado <david.cunado@arm.com>

Standard SMC requests that are handled in the secure-world by the Secure
Payload can be preempted by interrupts that must be handled in the
normal world. When the TSP is preempted the secure context is stored and
control is passed to the normal world to handle the non-secure
interrupt. Once completed the preempted secure context is restored. When
restoring the preempted context, the dispatcher assumes that the TSP
preempted context is still stored as the SECURE context by the context
management library.

However, PSCI power management operations causes synchronous entry into
TSP. This overwrites the preempted SECURE context in the context
management library. When restoring back the SECURE context, the Secure
Payload crashes because this context is not the preempted context
anymore.

This patch avoids corruption of the preempted SECURE context by aborting
any preempted SMC during PSCI power management calls. The
abort_std_smc_entry hook of the TSP is called when aborting the SMC
request.

It also exposes this feature as a FAST SMC callable from normal world to
abort preempted SMC with FID TSP_FID_ABORT.

Change-Id: I7a70347e9293f47d87b5de20484b4ffefb56b770
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>

Add a debug assertion in the initialization function of Trusty's SPD
to check for the presence of Trusty. If Trusty is absent then the SPD's
setup function already detects it and returns an error code so the init
function will never been called. Therefore, a debug assertion is enough
to catch this improbable error case.

Change-Id: Id20013e9291cdeef7827b919de2a22455f6cd9f9
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

This patch fixes the following coding style error reported
by the checkpatch.pl script:

  Bad function definition - void el3_exit() should probably
  be void el3_exit(void)

There is another one but it's a false positive so there's no
point in fixing it:

  space prohibited after that '&' (ctx:WxW)
  +#define SMC_NR(entity, fn, fastcall, smc64) ((((fastcall) & 0x1) << 31) | \
                                                            ^
Change-Id: I34de0337c7216dabd16395879f13845a60ee6df0
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

In release builds, the Trusty SPD fails to build because of an unused
variable. Note that this warning message doesn't show in debug builds
because INFO() messages are not compiled out like in release mode.

This patch fixes this issue by removing this variable and using its
value in place directly in the INFO() macro call.

Change-Id: I1f552421181a09412315eef4eaca586012022018
Signed-off-by: Sandrine Bailleux <sandrine.bailleux@arm.com>

This patch adds the secure payload dispatcher for interacting
with Google's Trusty TEE. Documentation for Trusty can be found
at https://source.android.com/security/trusty



Original authors:
-----------------
* Arve Hjønnevåg <arve@android.com>
* Michael Ryleev <gmar@google.com>
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

Without an explicit cache flush, the next timestamp captured might have
a bogus value.

This can happen if the following operations happen in order,
on a CPU that's being powered down.

1) ENTER PSCI timestamp is captured with caches enabled.

2) The next timestamp (ENTER_HW_LOW_PWR) is captured with caches
   disabled.

3) On a system that uses a write-back cache configuration, the
   cache line that holds the PMF timestamps is evicted.

After step 1), the ENTER_PSCI timestamp is cached and not in main memory.
After step 2), the ENTER_HW_LOW_PWR timestamp is stored in main memory.
Before the CPU power down happens, the hardware evicts the cache line that
contains the PMF timestamps for this service.  As a result, the timestamp
captured in step 2) is overwritten with a bogus value.

Change-Id: Ic1bd816498d1a6d4dc16540208ed3a5efe43f529
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

In order to quantify the overall time spent in the PSCI software
implementation, an initial collection of PMF instrumentation points
has been added.

Instrumentation has been added to the following code paths:

- Entry to PSCI SMC handler.  The timestamp is captured as early
  as possible during the runtime exception and stored in memory
  before entering the PSCI SMC handler.

- Exit from PSCI SMC handler.  The timestamp is captured after
  normal return from the PSCI SMC handler or if a low power state
  was requested it is captured in the bl31 warm boot path before
  return to normal world.

- Entry to low power state.  The timestamp is captured before entry
  to a low power state which implies either standby or power down.
  As these power states are mutually exclusive, only one timestamp
  is defined to describe both.  It is possible to differentiate between
  the two power states using the PSCI STAT interface.

- Exit from low power state.  The timestamp is captured after a standby
  or power up operation has completed.

To calculate the number of cycles spent running code in Trusted Firmware
one can perform the following calculation:

(exit_psci - enter_psci) - (exit_low_pwr - enter_low_pwr).

The resulting number of cycles can be converted to time given the
frequency of the counter.

Change-Id: Ie3b8f3d16409b6703747093b3a2d5c7429ad0166
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>

This patch moves the invocation of `psci_setup()` from BL31 and SP_MIN
into `std_svc_setup()` as part of ARM Standard Service initialization.
This allows us to consolidate ARM Standard Service initializations which
will be added to in the future. A new function `get_arm_std_svc_args()`
is introduced to get arguments corresponding to each standard service.
This function must be implemented by the EL3 Runtime Firmware and both
SP_MIN and BL31 implement it.

Change-Id: I38e1b644f797fa4089b20574bd4a10f0419de184

This patch introduces the PSCI Library interface. The major changes
introduced are as follows:

* Earlier BL31 was responsible for Architectural initialization during cold
boot via bl31_arch_setup() whereas PSCI was responsible for the same during
warm boot. This functionality is now consolidated by the PSCI library
and it does Architectural initialization via psci_arch_setup() during both
cold and warm boots.

* Earlier the warm boot entry point was always `psci_entrypoint()`. This was
not flexible enough as a library interface. Now PSCI expects the runtime
firmware to provide the entry point via `psci_setup()`. A new function
`bl31_warm_entrypoint` is introduced in BL31 and the previous
`psci_entrypoint()` is deprecated.

* The `smc_helpers.h` is reorganized to separate the SMC Calling Convention
defines from the Trusted Firmware SMC helpers. The former is now in a new
header file `smcc.h` and the SMC helpers are moved to Architecture specific
header.

* The CPU context is used by PSCI for context initialization and
restoration after power down (PSCI Context). It is also used by BL31 for SMC
handling and context management during Normal-Secure world switch (SMC
Context). The `psci_smc_handler()` interface is redefined to not use SMC
helper macros thus enabling to decouple the PSCI context from EL3 runtime
firmware SMC context. This enables PSCI to be integrated with other runtime
firmware using a different SMC context.

NOTE: With this patch the architectural setup done in `bl31_arch_setup()`
is done as part of `psci_setup()` and hence `bl31_platform_setup()` will be
invoked prior to architectural setup. It is highly unlikely that the platform
setup will depend on architectural setup and cause any failure. Please be
be aware of this change in sequence.

Change-Id: I7f497a08d33be234bbb822c28146250cb20dab73

This patch moves the PSCI services and BL31 frameworks like context
management and per-cpu data into new library components `PSCI` and
`el3_runtime` respectively. This enables PSCI to be built independently from
BL31. A new `psci_lib.mk` makefile is introduced which adds the relevant
PSCI library sources and gets included by `bl31.mk`. Other changes which
are done as part of this patch are:

* The runtime services framework is now moved to the `common/` folder to
  enable reuse.
* The `asm_macros.S` and `assert_macros.S` helpers are moved to architecture
  specific folder.
* The `plat_psci_common.c` is moved from the `plat/common/aarch64/` folder
  to `plat/common` folder. The original file location now has a stub which
  just includes the file from new location to maintain platform compatibility.

Most of the changes wouldn't affect platform builds as they just involve
changes to the generic bl1.mk and bl31.mk makefiles.

NOTE: THE `plat_psci_common.c` FILE HAS MOVED LOCATION AND THE STUB FILE AT
THE ORIGINAL LOCATION IS NOW DEPRECATED. PLATFORMS SHOULD MODIFY THEIR
MAKEFILES TO INCLUDE THE FILE FROM THE NEW LOCATION.

Change-Id: I6bd87d5b59424995c6a65ef8076d4fda91ad5e86

This patch fixes some coding guideline warnings reported by the checkpatch
script. Only files related to upcoming feature development have been fixed.

Change-Id: I26fbce75c02ed62f00493ed6c106fe7c863ddbc5

This patch reworks type usage in generic code, drivers and ARM platform files
to make it more portable. The major changes done with respect to
type usage are as listed below:

* Use uintptr_t for storing address instead of uint64_t or unsigned long.
* Review usage of unsigned long as it can no longer be assumed to be 64 bit.
* Use u_register_t for register values whose width varies depending on
  whether AArch64 or AArch32.
* Use generic C types where-ever possible.

In addition to the above changes, this patch also modifies format specifiers
in print invocations so that they are AArch64/AArch32 agnostic. Only files
related to upcoming feature development have been reworked.

Change-Id: I9f8c78347c5a52ba7027ff389791f1dad63ee5f8

This patch adds following optional PSCI STAT functions:

- PSCI_STAT_RESIDENCY: This call returns the amount of time spent
  in power_state in microseconds, by the node represented by the
  `target_cpu` and the highest level of `power_state`.

- PSCI_STAT_COUNT: This call returns the number of times a
  `power_state` has been used by the node represented by the
  `target_cpu` and the highest power level of `power_state`.

These APIs provides residency statistics for power states that has
been used by the platform. They are implemented according to v1.0
of the PSCI specification.

By default this optional feature is disabled in the PSCI
implementation. To enable it, set the boolean flag
`ENABLE_PSCI_STAT` to 1. This also sets `ENABLE_PMF` to 1.

Change-Id: Ie62e9d37d6d416ccb1813acd7f616d1ddd3e8aff

OPTEE to execute in aarch64 bit mode, set it accordingly
when execution transitions from EL3 to EL1

Change-Id: I59f2f940bdc1aac10543045b006a137d107ec95f
Signed-off-by: Ashutosh Singh <ashutosh.singh@arm.com>

This patch adds a new optional platform hook `pwr_domain_pwr_down_wfi()` in
the plat_psci_ops structure. This hook allows the platform to perform platform
specific actions including the wfi invocation to enter powerdown. This hook
is invoked by both psci_do_cpu_off() and psci_cpu_suspend_start() functions.
The porting-guide.md is also updated for the same.

This patch also modifies the `psci_power_down_wfi()` function to invoke
`plat_panic_handler` incase of panic instead of the busy while loop.

Fixes ARM-Software/tf-issues#375

Change-Id: Iba104469a1445ee8d59fb3a6fdd0a98e7f24dfa3

Added plat_get_syscnt_freq2, which is a 32 bit variant of the 64 bit
plat_get_syscnt_freq. The old one has been flagged as deprecated.
Common code has been updated to use this new version. Porting guide
has been updated.

Change-Id: I9e913544926c418970972bfe7d81ee88b4da837e

The "end power level" value passed as the 3rd argument to the
psci_cpu_on_start() function is not used so this patch removes it.

Change-Id: Icaa68b8c4ecd94507287970455fbff354faaa41e

This patch introduces some debug assertions in the function
psci_cpu_on_start() to check the arguments it receives are
valid.

Change-Id: If4d23c9f668fb46f2d18c5e2ed1929498cc6736b

In new communication protocol between optee os and linux driver,
r0-r6 registers are used. opteed need to copy these registers
as well when optee context registers are initialized.

Change-Id: Ifb47b73f847c61746cb58ea78411c1c71f208030
Signed-off-by: Ashutosh Singh <ashutosh.singh@arm.com>

When BL31 is compiled at `-O3` optimization level using Linaro GCC 4.9
AArch64 toolchain, it reports the following error:

```
services/std_svc/psci/psci_common.c: In function 'psci_do_state_coordination':
services/std_svc/psci/psci_common.c:220:27: error: array subscript is above
array bounds [-Werror=array-bounds]
  psci_req_local_pwr_states[pwrlvl - 1][cpu_idx] = req_pwr_state;
                           ^
```

This error is a false positive and this patch resolves the error by asserting
the array bounds in `psci_do_state_coordination()`.

Fixes ARM-software/tf-issues#347

Change-Id: I3584ed7b2e28faf455b082cb3281d6e1d11d6495

When a CPU is powered down using PSCI CPU OFF API, it disables its caches
and updates its `aff_info_state` to OFF. The corresponding cache line is
invalidated by the CPU so that the update will be observed by other CPUs
running with caches enabled. There is a possibility that another CPU
which has been trying to turn ON this CPU via PSCI CPU ON API,
has already seen the update to `aff_info_state` and proceeds to update
the state to ON_PENDING prior to the cache invalidation. This may result
in the update of the state to ON_PENDING being discarded.

This patch fixes this issue by making sure that the update of `aff_info_state`
to ON_PENDING sticks by reading back the value after the cache flush and
retrying it if not updated. The patch also adds a dsbish() to
`psci_do_cpu_off()` to ensure ordering of the update to `aff_info_state`
prior to cache line invalidation.

Fixes ARM-software/tf-issues#349

Change-Id: I225de99957fe89871f8c57bcfc243956e805dcca

Migrate all direct usage of __attribute__ to usage of their
corresponding macros from cdefs.h.
e.g.:
 - __attribute__((unused)) -> __unused
Signed-off-by: Soren Brinkmann <soren.brinkmann@xilinx.com>

Change-Id: I6f49bd779f2a4d577c6443dd160290656cdbc59b

This patch removes the dash character from the image name, to
follow the image terminology in the Trusted Firmware Wiki page:

    https://github.com/ARM-software/arm-trusted-firmware/wiki

Changes apply to output messages, comments and documentation.

non-ARM platform files have been left unmodified.

Change-Id: Ic2a99be4ed929d52afbeb27ac765ceffce46ed76

Earlier the TSP only ever expected to be preempted during Standard SMC
processing. If a S-EL1 interrupt triggered while in the normal world, it
will routed to S-EL1 `synchronously` for handling. The `synchronous` S-EL1
interrupt handler `tsp_sel1_intr_entry` used to panic if this S-EL1 interrupt
was preempted by another higher priority pending interrupt which should be
handled in EL3 e.g. Group0 interrupt in GICv3.

With this patch, the `tsp_sel1_intr_entry` now expects `TSP_PREEMPTED` as the
return code from the `tsp_common_int_handler` in addition to 0 (interrupt
successfully handled) and in both cases it issues an SMC with id
`TSP_HANDLED_S_EL1_INTR`. The TSPD switches the context and returns back
to normal world. In case a higher priority EL3 interrupt was pending, the
execution will be routed to EL3 where interrupt will be handled. On return
back to normal world, the pending S-EL1 interrupt which was preempted will
get routed to S-EL1 to be handled `synchronously` via `tsp_sel1_intr_entry`.

Change-Id: I2087c7fedb37746fbd9200cdda9b6dba93e16201

On a GICv2 system, interrupts that should be handled in the secure world are
typically signalled as FIQs. On a GICv3 system, these interrupts are signalled
as IRQs instead. The mechanism for handling both types of interrupts is the same
in both cases. This patch enables the TSP to run on a GICv3 system by:

1. adding support for handling IRQs in the exception handling code.
2. removing use of "fiq" in the names of data structures, macros and functions.

The build option TSPD_ROUTE_IRQ_TO_EL3 is deprecated and is replaced with a
new build flag TSP_NS_INTR_ASYNC_PREEMPT. For compatibility reasons, if the
former build flag is defined, it will be used to define the value for the
new build flag. The documentation is also updated accordingly.

Change-Id: I1807d371f41c3656322dd259340a57649833065e

The TSP is expected to pass control back to EL3 if it gets preempted due to
an interrupt while handling a Standard SMC in the following scenarios:

1. An FIQ preempts Standard SMC execution and that FIQ is not a TSP Secure
   timer interrupt or is preempted by a higher priority interrupt by the time
   the TSP acknowledges it. In this case, the TSP issues an SMC with the ID
   as `TSP_EL3_FIQ`. Currently this case is never expected to happen as only
   the TSP Secure Timer is expected to generate FIQ.

2. An IRQ preempts Standard SMC execution and in this case the TSP issues
   an SMC with the ID as `TSP_PREEMPTED`.

In both the cases, the TSPD hands control back to the normal world and returns
returns an error code to the normal world to indicate that the standard SMC it
had issued has been preempted but not completed.

This patch unifies the handling of these two cases in the TSPD and ensures that
the TSP only uses TSP_PREEMPTED instead of separate SMC IDs. Also instead of 2
separate error codes, SMC_PREEMPTED and TSP_EL3_FIQ, only SMC_PREEMPTED is
returned as error code back to the normal world.

Background information: On a GICv3 system, when the secure world has affinity
routing enabled, in 2. an FIQ will preempt TSP execution instead of an IRQ. The
FIQ could be a result of a Group 0 or a Group 1 NS interrupt. In both case, the
TSPD passes control back to the normal world upon receipt of the TSP_PREEMPTED
SMC. A Group 0 interrupt will immediately preempt execution to EL3 where it
will be handled. This allows for unified interrupt handling in TSP for both
GICv3 and GICv2 systems.

Change-Id: I9895344db74b188021e3f6a694701ad272fb40d4

The IMF_READ_INTERRUPT_ID build option enables a feature where the interrupt
ID of the highest priority pending interrupt is passed as a parameter to the
interrupt handler registered for that type of interrupt. This additional read
of highest pending interrupt id from GIC is problematic as it is possible that
the original interrupt may get deasserted and another interrupt of different
type maybe become the highest pending interrupt. Hence it is safer to prevent
such behaviour by removing the IMF_READ_INTERRUPT_ID build option.

The `id` parameter of the interrupt handler `interrupt_type_handler_t` is
now made a reserved parameter with this patch. It will always contain
INTR_ID_UNAVAILABLE.

Fixes ARM-software/tf-issues#307

Change-Id: I2173aae1dd37edad7ba6bdfb1a99868635fa34de

TLK sends the "preempted" event to the NS world along with an
identifier for certain use cases. The NS world driver is then
expected to take appropriate action depending on the identifier
value. Upon completion, the NS world driver then sends the
results to TLK (via x1-x3) with the TLK_RESUME_FID function ID.

This patch uses the already present code to pass the results
from the NS world to TLK for the TLK_RESUME_FID function ID.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

This patch fixes an issue in the PSCI framework where the affinity info
state of a core was being set to OFF even when the SPD had denied the
CPU_OFF request. Now, the state remains set to ON instead.

Fixes ARM-software/tf-issues#323

Change-Id: Ia9042aa41fae574eaa07fd2ce3f50cf8cae1b6fc

This patch adds PM handlers to TLKD for the system suspend/resume and
system poweroff/reset cases. TLK expects all SMCs through a single
handler, which then fork out into multiple handlers depending on the
SMC. We tap into the same single entrypoint by restoring the S-EL1
context before passing the PM event via register 'x0'. On completion
of the PM event, TLK sends a completion SMC and TLKD then moves on
with the PM process.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).

This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:

1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
   initialised. This ensures that any stale cache lines at any level of cache
   are removed.

2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
   visible to secondary CPUs using a cache clean operation by MVA.

3. Cache maintenance by set/way operations are only used prior to power down.

NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.

Fixes ARM-software/tf-issues#205

Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a