GitLab

When acquiring or releasing the power domain locks for a given CPU the
parent nodes are looked up by walking the up the PD tree list on both the
acquire and release path, only one set of lookups is needed. Fetch the
parent nodes first and pass this list into both the acquire and release
functions to avoid the double lookup.

This also allows us to not have to do this lookup after coherency has
been exited during the core power down sequence. The shared struct
psci_cpu_pd_nodes is not placed in coherent memory like is done
for psci_non_cpu_pd_nodes and doing so would negatively affect
performance. With this patch we remove the need to have it in coherent
memory by moving the access out of psci_release_pwr_domain_locks().
Signed-off-by: Andrew F. Davis <afd@ti.com>
Change-Id: I7b9cfa9d31148dea0f5e21091c8b45ef7fe4c4ab

Enforce full include path for includes. Deprecate old paths.

The following folders inside include/lib have been left unchanged:

- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}

The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).

For example, this patch had to be created because two headers were
called the same way: e0ea0928 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3 ("drivers: add tzc380 support").

This problem was introduced in commit 4ecca339

 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.

Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.

Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Initial Spectre Variant 1 mitigations (CVE-2017-5753).
A potential speculative data leak was found in PSCI code, this depends
on a non-robust implementation of the `plat_get_core_pos_by_mpidr()`
function. This is considered very low-risk. This patch adds a macro to
mitigate this. Note not all code paths could be analyzed with current
tools.

Add a macro which makes a variable 'speculation safe', using the
 __builtin_speculation_safe_value function of GCC and llvm. This will be
available in GCC 9, and is planned for llvm, but is not currently in
mainline GCC or llvm. In order to implement this mitigation the compiler
must support this builtin. Support is indicated by the
__HAVE_SPECULATION_SAFE_VALUE flag.

The -mtrack-speculation option maintains a 'tracker' register, which
determines if the processor is in false speculation at any point. This
adds instructions and increases code size, but avoids the performance
impact of a hard barrier.

Without the -mtrack-speculation option, __builtin_speculation_safe_value
expands to a

    ISB
    DSB SY

sequence after a conditional branch, before the
speculation safe variable is used. With -mtrack-speculation a

    CSEL tracker, tracker, XZR, [cond];
    AND safeval,tracker;
    CSDB

sequence is added instead, clearing the vulnerable variable by
AND'ing it with the tracker register, which is zero during speculative
execution. [cond] are the status flags which will only be true during
speculative execution. For more information on
__builtin_speculation_safe_value and the -mtrack-speculation option see
https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability/compiler-support-for-mitigations



The -mtracking option was not added, as the performance impact of the
mitigation is low, and there is only one occurence.

Change-Id: Ic9e66d1f4a5155e42e3e4055594974c230bfba3c
Signed-off-by: Joel Hutton <Joel.Hutton@Arm.com>

Some platforms can only resume from system suspend from the boot
CPU, hence they should only enter that state from that same core.

The following commit presents an interface that allows the platform to
reject system suspend entry near its very last stage (last CPU).

When a platform enables its caches before it accesses the
psci_non_cpu_pd_nodes structure then explicit cache maintenance
is not needed.
Signed-off-by: Andrew F. Davis <afd@ti.com>

The MMU is not disabled in this path, update the comment to
reflect this. Also clarify that both paths call prepare_cpu_pwr_dwn(),
but the second path does stack cache maintenance.
Signed-off-by: Andrew F. Davis <afd@ti.com>

Mark the initialization functions in BL31, such as context management,
EHF, RAS and PSCI as __init so that they can be reclaimed by the
platform when no longer needed

Change-Id: I7446aeee3dde8950b0f410cb766b7a2312c20130
Signed-off-by: Daniel Boulby <daniel.boulby@arm.com>

Change-Id: Icd1cdd42afdc78895a9be6c46b414b0a155cfa63
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Change-Id: I9fd8016527ad7706494f34356fdae8efacef5f72
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Change-Id: I40d040aa05bcbf11536a96ce59827711456b93a8
Co-authored-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

During cold boot, the initial translation tables are created with data
caches disabled, so all modifications go to memory directly. After the
MMU is enabled and data cache is enabled, any modification to the tables
goes to data cache, and eventually may get flushed to memory.

If CPU0 modifies the tables while CPU1 is off, CPU0 will have the
modified tables in its data cache. When CPU1 is powered on, the MMU is
enabled, then it enables coherency, and then it enables the data cache.
Until this is done, CPU1 isn't in coherency, and the translation tables
it sees can be outdated if CPU0 still has some modified entries in its
data cache.

This can be a problem in some cases. For example, the warm boot code
uses only the tables mapped during cold boot, which don't normally
change. However, if they are modified (and a RO page is made RW, or a XN
page is made executable) the CPU will see the old attributes and crash
when it tries to access it.

This doesn't happen in systems with HW_ASSISTED_COHERENCY or
WARMBOOT_ENABLE_DCACHE_EARLY. In these systems, the data cache is
enabled at the same time as the MMU. As soon as this happens, the CPU is
in coherency.

There was an attempt of a fix in psci_helpers.S, but it didn't solve the
problem. That code has been deleted. The code was introduced in commit
<26441030

> ("Invalidate TLB entries during warm boot").

Now, during a map or unmap operation, the memory associated to each
modified table is flushed. Traversing a table will also flush it's
memory, as there is no way to tell in the current implementation if the
table that has been traversed has also been modified.

Change-Id: I4b520bca27502f1018878061bc5fb82af740bb92
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Change-Id: I77c9cd2d1d6d0122cc49917fa686014bee154589
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

If either USE_COHERENT_MEM or HW_ASSISTED_COHERENCY being true
should cause us to not enter the ifdef block, then the logic
is not correct here. Posibly bad use of De Morgan's law?
Fix this.
Signed-off-by: Andrew F. Davis <afd@ti.com>

MISRA C-2012 Rules 10.1, 10.3, 17.8 and 20.7.

Change-Id: I3980bd2a1d845559af4bbe2887a0250d0506a064
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

MISRA C-2012 Rules 10.1, 10.3 and 20.7.

Change-Id: I972ce63f0d8fa157ed17e826b84f218fe498c517
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

MISRA C-2012 Rules 10.1 and 10.3.

Change-Id: I88cd5f56cda5780f2e0ba541c0f5b561309ab3af
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix violations of MISRA C-2012 Rules 8.13, 10.1, 10.3, 17.7 and 20.7.

Change-Id: I6f45a1069b742aebf9e1d6a403717b1522083f51
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Also change header guards to fix defects of MISRA C-2012 Rule 21.1.

Change-Id: Ied0d4b0e557ef6119ab669d106d2ac5d99620c57
Acked-by: Sumit Garg <sumit.garg@linaro.org>
Acked-by: Anson Huang <Anson.Huang@nxp.com>
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix MISRA C-2012 Directive 4.9 defects.

Change-Id: Ibd5364d8f138ddcf59c8074c32b35769366807dc
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Fix MISRA C-2012 Directive 4.9 and Rule 21.1 defects.

Change-Id: I96c216317d38741ee632d2640cd7b36e6723d5c2
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Use a _ prefix for Macro arguments to prevent that argument from
hiding variables of the same name in the outer scope

Rule 5.3: An identifier declared in an inner scope shall not
          hide an identifier declared in an outer scope

Fixed For:
    make LOG_LEVEL=50 PLAT=fvp

Change-Id: I67b6b05cbad4aeca65ce52981b4679b340604708
Signed-off-by: Daniel Boulby <daniel.boulby@arm.com>

Initializes each element of the last_cpu_in_non_cpu_pd array in PSCI
stat implementation to -1, the reset value. This satisfies MISRA rule
9.3.

Previously, only the first element of the array was initialized to -1.

Change-Id: I666c71e6c073710c67c6d24c07a219b1feb5b773
Signed-off-by: Jonathan Wright <jonathan.wright@arm.com>

Ensure (where possible) that switch statements in lib comply with MISRA
rules 16.1 - 16.7.

Change-Id: I52bc896fb7094d2b7569285686ee89f39f1ddd84
Signed-off-by: Jonathan Wright <jonathan.wright@arm.com>

When the source code says 'SMCC' it is talking about the SMC Calling
Convention. The correct acronym is SMCCC. This affects a few definitions
and file names.

Some files have been renamed (smcc.h, smcc_helpers.h and smcc_macros.S)
but the old files have been kept for compatibility, they include the
new ones with an ERROR_DEPRECATED guard.

Change-Id: I78f94052a502436fdd97ca32c0fe86bd58173f2f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

Rule 8.3: All declarations of an object or function shall
          use the same names and type qualifiers.

Change-Id: Iff384187c74a598a4e73f350a1893b60e9d16cec
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

During the warm boot sequence:

1. The MMU is enabled with the data cache disabled. The MMU table walker
   is set up to access the translation tables as in cacheable memory,
   but its accesses are non-cacheable because SCTLR_EL3.C controls them
   as well.
2. The interconnect is set up and the CPU enters coherency with the
   rest of the system.
3. The data cache is enabled.

If the support for dynamic translation tables is enabled and another CPU
makes changes to a region, the changes may only be present in the data
cache, not in RAM. The CPU that is booting isn't in coherency with the
rest of the system, so the table walker of that CPU isn't either. This
means that it may read old entries from RAM and it may have invalid TLB
entries corresponding to the dynamic mappings.

This is not a problem for the boot code because the mapping is 1:1 and
the regions are static. However, the code that runs after the boot
sequence may need to access the dynamically mapped regions.

This patch invalidates all TLBs during warm boot when the dynamic
translation tables support is enabled to prevent this problem.

Change-Id: I80264802dc0aa1cb3edd77d0b66b91db6961af3d
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>

On some platforms it may be necessary to discover the SMCCC version
via a PSCI features call.

Change-Id: I95281ac2263ca9aefda1809eb03464fbdb8ac24d
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

The suspend hook is published at the start of a CPU powerdown
operation.  The resume hook is published at the end of a CPU powerup
operation.

Change-Id: I50c05e2dde0d33834095ac41b4fcea4c161bb434
Signed-off-by: Dimitris Papastamos <dimitris.papastamos@arm.com>

There is an edge case where the cache maintaince done in
psci_do_cpu_off may not seen by some cores. This case is handled in
psci_cpu_on_start but it hasn't handled in psci_affinity_info.

Change-Id: I4d64f3d1ca9528e364aea8d04e2d254f201e1702
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

If ARMv7 based platform does not set ARM_CORTEX_Ax=yes, platform
shall define ARMV7_SUPPORTS_GENERIC_TIMER to enable generic timer
support.
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>

This patch includes various fixes for PSCI STAT functionality
relating to timestamp collection:

1. The PSCI stat accounting for retention states for higher level
power domains were done outside the locks which could lead to
spurious values in some race conditions. This is moved inside
the locks. Also, the call to start the stat accounting was redundant
which is now removed.

2. The timestamp wrap-around case when calculating residency did
not cater for AArch32. This is now fixed.

3. In the warm boot path, `plat_psci_stat_accounting_stop()` was
getting invoked prior to population of target power states. This
is now corrected.

Change-Id: I851526455304fb74ff0a724f4d5318cd89e19589
Signed-off-by: Soby Mathew <soby.mathew@arm.com>

This allows other EL3 components to subscribe to CPU on events.

Update Firmware Design guide to list psci_cpu_on_finish as an available
event.

Change-Id: Ida774afe0f9cdce4021933fcc33a9527ba7aaae2
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>

With this patch the PSCI_FEATURES API correctly reports availability
of the PSCI_MEM_PROTECT_CHECK API - PSCI_MEM_CHK_RANGE_AARCH64 is
added to the PSCI capabilities mask, PSCI_CAP_64BIT_MASK

Change-Id: Ic90ee804deaadf0f948dc2d46ac5fe4121ef77ae
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

This patch implements PSCI_SYSTEM_RESET2 API as defined in PSCI
v1.1 specification. The specification allows architectural and
vendor-specific resets via this API. In the current specification,
there is only one architectural reset, the warm reset. This reset is
intended to provide a fast reboot path that guarantees not to reset
system main memory.

Change-Id: I057bb81a60cd0fe56465dbb5791d8e1cca025bd3
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

This patch adds the generic code that links the psci smc handler
with the platform function that implements the mem_protect and
mem_check_range functionalities. These functions are  optional
APIs added in PSCI v1.1 (ARM DEN022D).

Change-Id: I3bac1307a5ce2c7a196ace76db8317e8d8c8bb3f
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

psci_set_power_off_state only initializes a local variable, so there
isn't any reason why it should be done while the lock is held.

Change-Id: I1c62f4cd5d860d102532e5a5350152180d41d127
Signed-off-by: Roberto Vargas <roberto.vargas@arm.com>

Previously commit 555ebb34db8f3424c1b394df2f10ecf9c1f70901 attmpted to fix this
GCC issue:

services/std_svc/psci/psci_common.c: In function 'psci_do_state_coordination':
services/std_svc/psci/psci_common.c:220:27: error: array subscript is above
array bounds [-Werror=array-bounds]
  psci_req_local_pwr_states[pwrlvl - 1][cpu_idx] = req_pwr_state;

This fix doesn't work as asserts aren't built in non-debug build flows.

Let's use GCCs #pragma option (documented here:
https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Pragmas.html

) to avoid
this false positive instead.
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>

This patch adds an early suspend handler, that executes with
SMP and data cache enabled. This handler allows platforms to
perform any early actions during the CPU suspend entry sequence.

This handler is optional and platforms can choose to implement it
depending on their needs. The `pwr_domain_suspend` handler still
exists and platforms can keep on using it without any side effects.
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>

There is a theoretical edge case during CPU_ON where the cache
may contain stale data for the target CPU data - this can occur
under the following conditions:

- the target CPU is in another cluster from the current
- the target CPU was the last CPU to shutdown on its cluster
- the cluster was removed from coherency as part of the CPU shutdown

In this case the cache maintenace that was performed as part of the
target CPUs shutdown was not seen by the current CPU's cluster. And
so the cache may contain stale data for the target CPU.

This patch adds a cache maintenance operation (flush) for the
cache-line containing the target CPU data - this ensures that the
target CPU data is read from main memory.

Change-Id: If8cfd42639b03174f60669429b7f7a757027d0fb
Signed-off-by: David Cunado <david.cunado@arm.com>

This fix modifies the order of system includes to meet the ARM TF coding
standard. There are some exceptions in order to retain header groupings,
minimise changes to imported headers, and where there are headers within
the #if and #ifndef statements.

Change-Id: I65085a142ba6a83792b26efb47df1329153f1624
Signed-off-by: Isla Mitchell <isla.mitchell@arm.com>