2. Initialise one serial port.
3. Detect the machine type.
4. Setup the kernel tagged list.
-5. Call the kernel image.
+5. Load initramfs.
+6. Call the kernel image.
1. Setup and initialise RAM
The boot loader must pass at a minimum the size and location of the
system memory, and the root filesystem location. The dtb must be
placed in a region of memory where the kernel decompressor will not
-overwrite it. The recommended placement is in the first 16KiB of RAM
-with the caveat that it may not be located at physical address 0 since
-the kernel interprets a value of 0 in r2 to mean neither a tagged list
-nor a dtb were passed.
+overwrite it, whilst remaining within the region which will be covered
+by the kernel's low-memory mapping.
-5. Calling the kernel image
+A safe location is just above the 128MiB boundary from start of RAM.
+
+5. Load initramfs.
+------------------
+
+Existing boot loaders: OPTIONAL
+New boot loaders: OPTIONAL
+
+If an initramfs is in use then, as with the dtb, it must be placed in
+a region of memory where the kernel decompressor will not overwrite it
+while also with the region which will be covered by the kernel's
+low-memory mapping.
+
+A safe location is just above the device tree blob which itself will
+be loaded just above the 128MiB boundary from the start of RAM as
+recommended above.
+
+6. Calling the kernel image
---------------------------
Existing boot loaders: MANDATORY
then it is legal for the boot loader to call the zImage in flash
directly.
-The zImage may also be placed in system RAM (at any location) and
-called there. Note that the kernel uses 16K of RAM below the image
-to store page tables. The recommended placement is 32KiB into RAM.
+The zImage may also be placed in system RAM and called there. The
+kernel should be placed in the first 128MiB of RAM. It is recommended
+that it is loaded above 32MiB in order to avoid the need to relocate
+prior to decompression, which will make the boot process slightly
+faster.
+
+When booting a raw (non-zImage) kernel the constraints are tighter.
+In this case the kernel must be loaded at an offset into system equal
+to TEXT_OFFSET - PAGE_OFFSET.
-In either case, the following conditions must be met:
+In any case, the following conditions must be met:
- Quiesce all DMA capable devices so that memory does not get
corrupted by bogus network packets or disk data. This will save
--- /dev/null
+Kernel mode NEON
+================
+
+TL;DR summary
+-------------
+* Use only NEON instructions, or VFP instructions that don't rely on support
+ code
+* Isolate your NEON code in a separate compilation unit, and compile it with
+ '-mfpu=neon -mfloat-abi=softfp'
+* Put kernel_neon_begin() and kernel_neon_end() calls around the calls into your
+ NEON code
+* Don't sleep in your NEON code, and be aware that it will be executed with
+ preemption disabled
+
+
+Introduction
+------------
+It is possible to use NEON instructions (and in some cases, VFP instructions) in
+code that runs in kernel mode. However, for performance reasons, the NEON/VFP
+register file is not preserved and restored at every context switch or taken
+exception like the normal register file is, so some manual intervention is
+required. Furthermore, special care is required for code that may sleep [i.e.,
+may call schedule()], as NEON or VFP instructions will be executed in a
+non-preemptible section for reasons outlined below.
+
+
+Lazy preserve and restore
+-------------------------
+The NEON/VFP register file is managed using lazy preserve (on UP systems) and
+lazy restore (on both SMP and UP systems). This means that the register file is
+kept 'live', and is only preserved and restored when multiple tasks are
+contending for the NEON/VFP unit (or, in the SMP case, when a task migrates to
+another core). Lazy restore is implemented by disabling the NEON/VFP unit after
+every context switch, resulting in a trap when subsequently a NEON/VFP
+instruction is issued, allowing the kernel to step in and perform the restore if
+necessary.
+
+Any use of the NEON/VFP unit in kernel mode should not interfere with this, so
+it is required to do an 'eager' preserve of the NEON/VFP register file, and
+enable the NEON/VFP unit explicitly so no exceptions are generated on first
+subsequent use. This is handled by the function kernel_neon_begin(), which
+should be called before any kernel mode NEON or VFP instructions are issued.
+Likewise, the NEON/VFP unit should be disabled again after use to make sure user
+mode will hit the lazy restore trap upon next use. This is handled by the
+function kernel_neon_end().
+
+
+Interruptions in kernel mode
+----------------------------
+For reasons of performance and simplicity, it was decided that there shall be no
+preserve/restore mechanism for the kernel mode NEON/VFP register contents. This
+implies that interruptions of a kernel mode NEON section can only be allowed if
+they are guaranteed not to touch the NEON/VFP registers. For this reason, the
+following rules and restrictions apply in the kernel:
+* NEON/VFP code is not allowed in interrupt context;
+* NEON/VFP code is not allowed to sleep;
+* NEON/VFP code is executed with preemption disabled.
+
+If latency is a concern, it is possible to put back to back calls to
+kernel_neon_end() and kernel_neon_begin() in places in your code where none of
+the NEON registers are live. (Additional calls to kernel_neon_begin() should be
+reasonably cheap if no context switch occurred in the meantime)
+
+
+VFP and support code
+--------------------
+Earlier versions of VFP (prior to version 3) rely on software support for things
+like IEEE-754 compliant underflow handling etc. When the VFP unit needs such
+software assistance, it signals the kernel by raising an undefined instruction
+exception. The kernel responds by inspecting the VFP control registers and the
+current instruction and arguments, and emulates the instruction in software.
+
+Such software assistance is currently not implemented for VFP instructions
+executed in kernel mode. If such a condition is encountered, the kernel will
+fail and generate an OOPS.
+
+
+Separating NEON code from ordinary code
+---------------------------------------
+The compiler is not aware of the special significance of kernel_neon_begin() and
+kernel_neon_end(), i.e., that it is only allowed to issue NEON/VFP instructions
+between calls to these respective functions. Furthermore, GCC may generate NEON
+instructions of its own at -O3 level if -mfpu=neon is selected, and even if the
+kernel is currently compiled at -O2, future changes may result in NEON/VFP
+instructions appearing in unexpected places if no special care is taken.
+
+Therefore, the recommended and only supported way of using NEON/VFP in the
+kernel is by adhering to the following rules:
+* isolate the NEON code in a separate compilation unit and compile it with
+ '-mfpu=neon -mfloat-abi=softfp';
+* issue the calls to kernel_neon_begin(), kernel_neon_end() as well as the calls
+ into the unit containing the NEON code from a compilation unit which is *not*
+ built with the GCC flag '-mfpu=neon' set.
+
+As the kernel is compiled with '-msoft-float', the above will guarantee that
+both NEON and VFP instructions will only ever appear in designated compilation
+units at any optimization level.
+
+
+NEON assembler
+--------------
+NEON assembler is supported with no additional caveats as long as the rules
+above are followed.
+
+
+NEON code generated by GCC
+--------------------------
+The GCC option -ftree-vectorize (implied by -O3) tries to exploit implicit
+parallelism, and generates NEON code from ordinary C source code. This is fully
+supported as long as the rules above are followed.
+
+
+NEON intrinsics
+---------------
+NEON intrinsics are also supported. However, as code using NEON intrinsics
+relies on the GCC header <arm_neon.h>, (which #includes <stdint.h>), you should
+observe the following in addition to the rules above:
+* Compile the unit containing the NEON intrinsics with '-ffreestanding' so GCC
+ uses its builtin version of <stdint.h> (this is a C99 header which the kernel
+ does not supply);
+* Include <arm_neon.h> last, or at least after <linux/types.h>
performs the same operation).
"marvell,"aurora-outer-cache: Marvell Controller designed to be
compatible with the ARM one with outer cache mode.
- "bcm,bcm11351-a2-pl310-cache": For Broadcom bcm11351 chipset where an
+ "brcm,bcm11351-a2-pl310-cache": For Broadcom bcm11351 chipset where an
offset needs to be added to the address before passing down to the L2
cache controller
+ "bcm,bcm11351-a2-pl310-cache": DEPRECATED by
+ "brcm,bcm11351-a2-pl310-cache"
- cache-unified : Specifies the cache is a unified cache.
- cache-level : Should be set to 2 for a level 2 cache.
- reg : Physical base address and size of cache controller's memory mapped
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
+ select IRQ_FORCED_THREADING
select KTIME_SCALAR
select PERF_USE_VMALLOC
select RTC_LIB
which sends an IPI to the CPUs that are running the same ASID
as the one being invalidated.
+config ARM_ERRATA_773022
+ bool "ARM errata: incorrect instructions may be executed from loop buffer"
+ depends on CPU_V7
+ help
+ This option enables the workaround for the 773022 Cortex-A15
+ (up to r0p4) erratum. In certain rare sequences of code, the
+ loop buffer may deliver incorrect instructions. This
+ workaround disables the loop buffer to avoid the erratum.
+
endmenu
source "arch/arm/common/Kconfig"
source kernel/Kconfig.preempt
-config HZ
+config HZ_FIXED
int
default 200 if ARCH_EBSA110 || ARCH_S3C24XX || ARCH_S5P64X0 || \
ARCH_S5PV210 || ARCH_EXYNOS4
default AT91_TIMER_HZ if ARCH_AT91
default SHMOBILE_TIMER_HZ if ARCH_SHMOBILE
- default 100
+
+choice
+ depends on !HZ_FIXED
+ prompt "Timer frequency"
+
+config HZ_100
+ bool "100 Hz"
+
+config HZ_200
+ bool "200 Hz"
+
+config HZ_250
+ bool "250 Hz"
+
+config HZ_300
+ bool "300 Hz"
+
+config HZ_500
+ bool "500 Hz"
+
+config HZ_1000
+ bool "1000 Hz"
+
+endchoice
+
+config HZ
+ int
+ default HZ_FIXED if HZ_FIXED
+ default 100 if HZ_100
+ default 200 if HZ_200
+ default 250 if HZ_250
+ default 300 if HZ_300
+ default 500 if HZ_500
+ default 1000
+
+config SCHED_HRTICK
+ def_bool HIGH_RES_TIMERS
config SCHED_HRTICK
def_bool HIGH_RES_TIMERS
def_bool y
depends on ARM_LPAE
+config ARCH_WANT_GENERAL_HUGETLB
+ def_bool y
+
source "mm/Kconfig"
config FORCE_MAX_ZONEORDER
Say Y to include support code for NEON, the ARMv7 Advanced SIMD
Extension.
+config KERNEL_MODE_NEON
+ bool "Support for NEON in kernel mode"
+ default n
+ depends on NEON
+ help
+ Say Y to include support for NEON in kernel mode.
+
endmenu
menu "Userspace binary formats"
config ARCH_SUSPEND_POSSIBLE
depends on !ARCH_S5PC100
- depends on CPU_ARM920T || CPU_ARM926T || CPU_SA1100 || \
+ depends on CPU_ARM920T || CPU_ARM926T || CPU_FEROCEON || CPU_SA1100 || \
CPU_V6 || CPU_V6K || CPU_V7 || CPU_XSC3 || CPU_XSCALE || CPU_MOHAWK
def_bool y
config DEBUG_BCM2835
bool "Kernel low-level debugging on BCM2835 PL011 UART"
depends on ARCH_BCM2835
+ select DEBUG_UART_PL01X
config DEBUG_CLPS711X_UART1
bool "Kernel low-level debugging messages via UART1"
config DEBUG_CNS3XXX
bool "Kernel Kernel low-level debugging on Cavium Networks CNS3xxx"
depends on ARCH_CNS3XXX
+ select DEBUG_UART_PL01X
help
Say Y here if you want the debug print routines to direct
their output to the CNS3xxx UART0.
config DEBUG_DAVINCI_DA8XX_UART1
bool "Kernel low-level debugging on DaVinci DA8XX using UART1"
depends on ARCH_DAVINCI_DA8XX
+ select DEBUG_UART_8250
help
Say Y here if you want the debug print routines to direct
their output to UART1 serial port on DaVinci DA8XX devices.
config DEBUG_DAVINCI_DA8XX_UART2
bool "Kernel low-level debugging on DaVinci DA8XX using UART2"
depends on ARCH_DAVINCI_DA8XX
+ select DEBUG_UART_8250
help
Say Y here if you want the debug print routines to direct
their output to UART2 serial port on DaVinci DA8XX devices.
config DEBUG_DAVINCI_DMx_UART0
bool "Kernel low-level debugging on DaVinci DMx using UART0"
depends on ARCH_DAVINCI_DMx
+ select DEBUG_UART_8250
help
Say Y here if you want the debug print routines to direct
their output to UART0 serial port on DaVinci DMx devices.
config DEBUG_DAVINCI_TNETV107X_UART1
bool "Kernel low-level debugging on DaVinci TNETV107x using UART1"
depends on ARCH_DAVINCI_TNETV107X
+ select DEBUG_UART_8250
help
Say Y here if you want the debug print routines to direct
their output to UART1 serial port on DaVinci TNETV107X
Say Y here if you want the debug print routines to direct
their output to the 8250 at PCI COM1.
+ config DEBUG_HI3620_UART
+ bool "Hisilicon HI3620 Debug UART"
+ depends on ARCH_HI3xxx
+ select DEBUG_UART_PL01X
+ help
+ Say Y here if you want kernel low-level debugging support
+ on HI3620 UART.
+
+ config DEBUG_HI3716_UART
+ bool "Hisilicon Hi3716 Debug UART"
+ depends on ARCH_HI3xxx
+ select DEBUG_UART_PL01X
+ help
+ Say Y here if you want kernel low-level debugging support
+ on HI3716 UART.
+
config DEBUG_HIGHBANK_UART
bool "Kernel low-level debugging messages via Highbank UART"
depends on ARCH_HIGHBANK
+ select DEBUG_UART_PL01X
help
Say Y here if you want the debug print routines to direct
their output to the UART on Highbank based devices.
config DEBUG_IMX23_UART
bool "i.MX23 Debug UART"
depends on SOC_IMX23
+ select DEBUG_UART_PL01X
help
Say Y here if you want kernel low-level debugging support
on i.MX23.
config DEBUG_IMX28_UART
bool "i.MX28 Debug UART"
depends on SOC_IMX28
+ select DEBUG_UART_PL01X
help
Say Y here if you want kernel low-level debugging support
on i.MX28.
config DEBUG_KEYSTONE_UART0
bool "Kernel low-level debugging on KEYSTONE2 using UART0"
depends on ARCH_KEYSTONE
+ select DEBUG_UART_8250
help
Say Y here if you want the debug print routines to direct
their output to UART0 serial port on KEYSTONE2 devices.
config DEBUG_KEYSTONE_UART1
bool "Kernel low-level debugging on KEYSTONE2 using UART1"
depends on ARCH_KEYSTONE
+ select DEBUG_UART_8250
help
Say Y here if you want the debug print routines to direct
their output to UART1 serial port on KEYSTONE2 devices.
config DEBUG_MMP_UART2
bool "Kernel low-level debugging message via MMP UART2"
depends on ARCH_MMP
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on MMP UART2.
config DEBUG_MMP_UART3
bool "Kernel low-level debugging message via MMP UART3"
depends on ARCH_MMP
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on MMP UART3.
config DEBUG_MVEBU_UART
bool "Kernel low-level debugging messages via MVEBU UART (old bootloaders)"
depends on ARCH_MVEBU
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on MVEBU based platforms.
config DEBUG_MVEBU_UART_ALTERNATE
bool "Kernel low-level debugging messages via MVEBU UART (new bootloaders)"
depends on ARCH_MVEBU
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on MVEBU based platforms.
config DEBUG_NOMADIK_UART
bool "Kernel low-level debugging messages via NOMADIK UART"
depends on ARCH_NOMADIK
+ select DEBUG_UART_PL01X
help
Say Y here if you want kernel low-level debugging support
on NOMADIK based platforms.
config DEBUG_NSPIRE_CLASSIC_UART
bool "Kernel low-level debugging via TI-NSPIRE 8250 UART"
depends on ARCH_NSPIRE
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on TI-NSPIRE classic models.
config DEBUG_NSPIRE_CX_UART
bool "Kernel low-level debugging via TI-NSPIRE PL011 UART"
depends on ARCH_NSPIRE
+ select DEBUG_UART_PL01X
help
Say Y here if you want kernel low-level debugging support
on TI-NSPIRE CX models.
- config DEBUG_OMAP2PLUS_UART
- bool "Kernel low-level debugging messages via OMAP2PLUS UART"
+ config DEBUG_OMAP2UART1
+ bool "OMAP2/3/4 UART1 (omap2/3 sdp boards and some omap3 boards)"
depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
help
- Say Y here if you want kernel low-level debugging support
- on OMAP2PLUS based platforms.
+ This covers at least h4, 2430sdp, 3430sdp, 3630sdp,
+ omap3 torpedo and 3530 lv som.
+
+ config DEBUG_OMAP2UART2
+ bool "Kernel low-level debugging messages via OMAP2/3/4 UART2"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_OMAP2UART3
+ bool "Kernel low-level debugging messages via OMAP2 UART3 (n8x0)"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_OMAP3UART3
+ bool "Kernel low-level debugging messages via OMAP3 UART3 (most omap3 boards)"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+ help
+ This covers at least cm_t3x, beagle, crane, devkit8000,
+ igep00x0, ldp, n900, n9(50), pandora, overo, touchbook,
+ and 3517evm.
+
+ config DEBUG_OMAP4UART3
+ bool "Kernel low-level debugging messages via OMAP4/5 UART3 (omap4 blaze, panda, omap5 sevm)"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_OMAP3UART4
+ bool "Kernel low-level debugging messages via OMAP36XX UART4"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_OMAP4UART4
+ bool "Kernel low-level debugging messages via OMAP4/5 UART4"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_TI81XXUART1
+ bool "Kernel low-level debugging messages via TI81XX UART1 (ti8148evm)"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_TI81XXUART2
+ bool "Kernel low-level debugging messages via TI81XX UART2"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_TI81XXUART3
+ bool "Kernel low-level debugging messages via TI81XX UART3 (ti8168evm)"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_AM33XXUART1
+ bool "Kernel low-level debugging messages via AM33XX UART1"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
+
+ config DEBUG_ZOOM_UART
+ bool "Kernel low-level debugging messages via Zoom2/3 UART"
+ depends on ARCH_OMAP2PLUS
+ select DEBUG_OMAP2PLUS_UART
config DEBUG_PICOXCELL_UART
depends on ARCH_PICOXCELL
bool "Use PicoXcell UART for low-level debug"
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on PicoXcell based platforms.
config DEBUG_PXA_UART1
depends on ARCH_PXA
bool "Use PXA UART1 for low-level debug"
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on PXA UART1.
config DEBUG_REALVIEW_STD_PORT
bool "RealView Default UART"
depends on ARCH_REALVIEW
+ select DEBUG_UART_PL01X
help
Say Y here if you want the debug print routines to direct
their output to the serial port on RealView EB, PB11MP, PBA8
config DEBUG_REALVIEW_PB1176_PORT
bool "RealView PB1176 UART"
depends on MACH_REALVIEW_PB1176
+ select DEBUG_UART_PL01X
help
Say Y here if you want the debug print routines to direct
their output to the standard serial port on the RealView
PB1176 platform.
- config DEBUG_ROCKCHIP_UART
- bool "Kernel low-level debugging messages via Rockchip UART"
+ config DEBUG_RK29_UART0
+ bool "Kernel low-level debugging messages via Rockchip RK29 UART0"
+ depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Rockchip based platforms.
+
+ config DEBUG_RK29_UART1
+ bool "Kernel low-level debugging messages via Rockchip RK29 UART1"
+ depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Rockchip based platforms.
+
+ config DEBUG_RK29_UART2
+ bool "Kernel low-level debugging messages via Rockchip RK29 UART2"
+ depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Rockchip based platforms.
+
+ config DEBUG_RK3X_UART0
+ bool "Kernel low-level debugging messages via Rockchip RK3X UART0"
+ depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Rockchip based platforms.
+
+ config DEBUG_RK3X_UART1
+ bool "Kernel low-level debugging messages via Rockchip RK3X UART1"
+ depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Rockchip based platforms.
+
+ config DEBUG_RK3X_UART2
+ bool "Kernel low-level debugging messages via Rockchip RK3X UART2"
+ depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Rockchip based platforms.
+
+ config DEBUG_RK3X_UART3
+ bool "Kernel low-level debugging messages via Rockchip RK3X UART3"
depends on ARCH_ROCKCHIP
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on Rockchip based platforms.
config DEBUG_SOCFPGA_UART
depends on ARCH_SOCFPGA
bool "Use SOCFPGA UART for low-level debug"
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on SOCFPGA based platforms.
config DEBUG_SUNXI_UART0
bool "Kernel low-level debugging messages via sunXi UART0"
depends on ARCH_SUNXI
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on Allwinner A1X based platforms on the UART0.
config DEBUG_SUNXI_UART1
bool "Kernel low-level debugging messages via sunXi UART1"
depends on ARCH_SUNXI
+ select DEBUG_UART_8250
help
Say Y here if you want kernel low-level debugging support
on Allwinner A1X based platforms on the UART1.
- config DEBUG_TEGRA_UART
+ config TEGRA_DEBUG_UART_AUTO_ODMDATA
+ bool "Kernel low-level debugging messages via Tegra UART via ODMDATA"
+ depends on ARCH_TEGRA
+ select DEBUG_TEGRA_UART
+ help
+ Automatically determines which UART to use for low-level
+ debug based on the ODMDATA value. This value is part of
+ the BCT, and is written to the boot memory device using
+ nvflash, or other flashing tool. When bits 19:18 are 3,
+ then bits 17:15 indicate which UART to use; 0/1/2/3/4
+ are UART A/B/C/D/E.
+
+ config TEGRA_DEBUG_UARTA
+ bool "Kernel low-level debugging messages via Tegra UART A"
+ depends on ARCH_TEGRA
+ select DEBUG_TEGRA_UART
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Tegra based platforms.
+
+ config TEGRA_DEBUG_UARTB
+ bool "Kernel low-level debugging messages via Tegra UART B"
+ depends on ARCH_TEGRA
+ select DEBUG_TEGRA_UART
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Tegra based platforms.
+
+ config TEGRA_DEBUG_UARTC
+ bool "Kernel low-level debugging messages via Tegra UART C"
+ depends on ARCH_TEGRA
+ select DEBUG_TEGRA_UART
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Tegra based platforms.
+
+ config TEGRA_DEBUG_UARTD
+ bool "Kernel low-level debugging messages via Tegra UART D"
+ depends on ARCH_TEGRA
+ select DEBUG_TEGRA_UART
+ help
+ Say Y here if you want kernel low-level debugging support
+ on Tegra based platforms.
+
+ config TEGRA_DEBUG_UARTE
+ bool "Kernel low-level debugging messages via Tegra UART E"
depends on ARCH_TEGRA
- bool "Use Tegra UART for low-level debug"
+ select DEBUG_TEGRA_UART
help
Say Y here if you want kernel low-level debugging support
on Tegra based platforms.
Say Y here if you want the debug print routines to direct
their output to the uart1 port on SiRFmarco devices.
- config DEBUG_STI_UART
+ config STIH41X_DEBUG_ASC2
+ bool "Use StiH415/416 ASC2 UART for low-level debug"
+ depends on ARCH_STI
+ select DEBUG_STI_UART
+ help
+ Say Y here if you want kernel low-level debugging support
+ on STiH415/416 based platforms like b2000, which has
+ default UART wired up to ASC2.
+
+ If unsure, say N.
+
+ config STIH41X_DEBUG_SBC_ASC1
+ bool "Use StiH415/416 SBC ASC1 UART for low-level debug"
depends on ARCH_STI
- bool "Use StiH415/416 ASC for low-level debug"
+ select DEBUG_STI_UART
help
Say Y here if you want kernel low-level debugging support
- on StiH415/416 based platforms like B2000, B2020.
- It support UART2 and SBC_UART1.
+ on STiH415/416 based platforms like b2020. which has
+ default UART wired up to SBC ASC1.
If unsure, say N.
config DEBUG_U300_UART
bool "Kernel low-level debugging messages via U300 UART0"
depends on ARCH_U300
+ select DEBUG_UART_PL01X
help
Say Y here if you want the debug print routines to direct
their output to the uart port on U300 devices.
config DEBUG_VEXPRESS_UART0_CA9
bool "Use PL011 UART0 at 0x10009000 (V2P-CA9 core tile)"
depends on ARCH_VEXPRESS
+ select DEBUG_UART_PL01X
help
This option selects UART0 at 0x10009000. Except for custom models,
this applies only to the V2P-CA9 tile.
config DEBUG_VEXPRESS_UART0_RS1
bool "Use PL011 UART0 at 0x1c090000 (RS1 complaint tiles)"
depends on ARCH_VEXPRESS
+ select DEBUG_UART_PL01X
help
This option selects UART0 at 0x1c090000. This applies to most
of the tiles using the RS1 memory map, including all new A-class
config DEBUG_VEXPRESS_UART0_CRX
bool "Use PL011 UART0 at 0xb0090000 (Cortex-R compliant tiles)"
depends on ARCH_VEXPRESS && !MMU
+ select DEBUG_UART_PL01X
help
This option selects UART0 at 0xb0090000. This is appropriate for
Cortex-R series tiles and SMMs, such as Cortex-R5 and Cortex-R7
depends on !ARCH_MULTIPLATFORM
help
Say Y here if your platform doesn't provide a UART option
- below. This relies on your platform choosing the right UART
+ above. This relies on your platform choosing the right UART
definition internally in order for low-level debugging to
work.
For more details about semihosting, please see
chapter 8 of DUI0203I_rvct_developer_guide.pdf from ARM Ltd.
+ config DEBUG_LL_UART_8250
+ bool "Kernel low-level debugging via 8250 UART"
+ help
+ Say Y here if you wish the debug print routes to direct
+ their output to an 8250 UART. You can use this option
+ to provide the parameters for the 8250 UART rather than
+ selecting one of the platform specific options above if
+ you know the parameters for the port.
+
+ This option is preferred over the platform specific
+ options; the platform specific options are deprecated
+ and will be soon removed.
+
+ config DEBUG_LL_UART_PL01X
+ bool "Kernel low-level debugging via ARM Ltd PL01x Primecell UART"
+ help
+ Say Y here if you wish the debug print routes to direct
+ their output to a PL01x Primecell UART. You can use
+ this option to provide the parameters for the UART
+ rather than selecting one of the platform specific
+ options above if you know the parameters for the port.
+
+ This option is preferred over the platform specific
+ options; the platform specific options are deprecated
+ and will be soon removed.
+
endchoice
config DEBUG_EXYNOS_UART
bool
+config DEBUG_OMAP2PLUS_UART
+ bool
+ depends on ARCH_OMAP2PLUS
+
config DEBUG_IMX_UART_PORT
int "i.MX Debug UART Port Selection" if DEBUG_IMX1_UART || \
DEBUG_IMX25_UART || \
Choose UART port on which kernel low-level debug messages
should be output.
-choice
- prompt "Low-level debug console UART"
- depends on DEBUG_OMAP2PLUS_UART
-
- config DEBUG_OMAP2UART1
- bool "OMAP2/3/4 UART1 (omap2/3 sdp boards and some omap3 boards)"
- help
- This covers at least h4, 2430sdp, 3430sdp, 3630sdp,
- omap3 torpedo and 3530 lv som.
-
- config DEBUG_OMAP2UART2
- bool "OMAP2/3/4 UART2"
-
- config DEBUG_OMAP2UART3
- bool "OMAP2 UART3 (n8x0)"
-
- config DEBUG_OMAP3UART3
- bool "OMAP3 UART3 (most omap3 boards)"
- help
- This covers at least cm_t3x, beagle, crane, devkit8000,
- igep00x0, ldp, n900, n9(50), pandora, overo, touchbook,
- and 3517evm.
-
- config DEBUG_OMAP4UART3
- bool "OMAP4/5 UART3 (omap4 blaze, panda, omap5 sevm)"
-
- config DEBUG_OMAP3UART4
- bool "OMAP36XX UART4"
-
- config DEBUG_OMAP4UART4
- bool "OMAP4/5 UART4"
-
- config DEBUG_TI81XXUART1
- bool "TI81XX UART1 (ti8148evm)"
-
- config DEBUG_TI81XXUART2
- bool "TI81XX UART2"
-
- config DEBUG_TI81XXUART3
- bool "TI81XX UART3 (ti8168evm)"
-
- config DEBUG_AM33XXUART1
- bool "AM33XX UART1"
-
- config DEBUG_ZOOM_UART
- bool "Zoom2/3 UART"
-endchoice
-
-choice
- prompt "Low-level debug console UART"
- depends on DEBUG_ROCKCHIP_UART
-
- config DEBUG_RK29_UART0
- bool "RK29 UART0"
-
- config DEBUG_RK29_UART1
- bool "RK29 UART1"
-
- config DEBUG_RK29_UART2
- bool "RK29 UART2"
-
- config DEBUG_RK3X_UART0
- bool "RK3X UART0"
-
- config DEBUG_RK3X_UART1
- bool "RK3X UART1"
-
- config DEBUG_RK3X_UART2
- bool "RK3X UART2"
-
- config DEBUG_RK3X_UART3
- bool "RK3X UART3"
-endchoice
-
-choice
- prompt "Low-level debug console UART"
- depends on DEBUG_LL && DEBUG_TEGRA_UART
-
- config TEGRA_DEBUG_UART_AUTO_ODMDATA
- bool "Via ODMDATA"
- help
- Automatically determines which UART to use for low-level debug based
- on the ODMDATA value. This value is part of the BCT, and is written
- to the boot memory device using nvflash, or other flashing tool.
- When bits 19:18 are 3, then bits 17:15 indicate which UART to use;
- 0/1/2/3/4 are UART A/B/C/D/E.
-
- config TEGRA_DEBUG_UARTA
- bool "UART A"
-
- config TEGRA_DEBUG_UARTB
- bool "UART B"
-
- config TEGRA_DEBUG_UARTC
- bool "UART C"
-
- config TEGRA_DEBUG_UARTD
- bool "UART D"
-
- config TEGRA_DEBUG_UARTE
- bool "UART E"
-
-endchoice
-
-choice
- prompt "Low-level debug console UART"
- depends on DEBUG_LL && DEBUG_STI_UART
-
- config STIH41X_DEBUG_ASC2
- bool "ASC2 UART"
- help
- Say Y here if you want kernel low-level debugging support
- on STiH415/416 based platforms like b2000, which has
- default UART wired up to ASC2.
-
- If unsure, say N.
-
- config STIH41X_DEBUG_SBC_ASC1
- bool "SBC ASC1 UART"
- help
- Say Y here if you want kernel low-level debugging support
- on STiH415/416 based platforms like b2020. which has
- default UART wired up to SBC ASC1.
-
- If unsure, say N.
+config DEBUG_TEGRA_UART
+ bool
+ depends on ARCH_TEGRA
-endchoice
+config DEBUG_STI_UART
+ bool
+ depends on ARCH_STI
config DEBUG_LL_INCLUDE
string
- default "debug/bcm2835.S" if DEBUG_BCM2835
- default "debug/cns3xxx.S" if DEBUG_CNS3XXX
+ default "debug/8250.S" if DEBUG_LL_UART_8250 || DEBUG_UART_8250
+ default "debug/pl01x.S" if DEBUG_LL_UART_PL01X || DEBUG_UART_PL01X
default "debug/exynos.S" if DEBUG_EXYNOS_UART
- default "debug/highbank.S" if DEBUG_HIGHBANK_UART
default "debug/icedcc.S" if DEBUG_ICEDCC
default "debug/imx.S" if DEBUG_IMX1_UART || \
DEBUG_IMX25_UART || \
DEBUG_IMX53_UART ||\
DEBUG_IMX6Q_UART || \
DEBUG_IMX6SL_UART
- default "debug/keystone.S" if DEBUG_KEYSTONE_UART0 || \
- DEBUG_KEYSTONE_UART1
- default "debug/mvebu.S" if DEBUG_MVEBU_UART || \
- DEBUG_MVEBU_UART_ALTERNATE
- default "debug/mxs.S" if DEBUG_IMX23_UART || DEBUG_IMX28_UART
- default "debug/nomadik.S" if DEBUG_NOMADIK_UART
- default "debug/nspire.S" if DEBUG_NSPIRE_CX_UART || \
- DEBUG_NSPIRE_CLASSIC_UART
default "debug/omap2plus.S" if DEBUG_OMAP2PLUS_UART
- default "debug/picoxcell.S" if DEBUG_PICOXCELL_UART
- default "debug/pxa.S" if DEBUG_PXA_UART1 || DEBUG_MMP_UART2 || \
- DEBUG_MMP_UART3
- default "debug/rockchip.S" if DEBUG_ROCKCHIP_UART
default "debug/sirf.S" if DEBUG_SIRFPRIMA2_UART1 || DEBUG_SIRFMARCO_UART1
- default "debug/socfpga.S" if DEBUG_SOCFPGA_UART
default "debug/sti.S" if DEBUG_STI_UART
- default "debug/sunxi.S" if DEBUG_SUNXI_UART0 || DEBUG_SUNXI_UART1
default "debug/tegra.S" if DEBUG_TEGRA_UART
- default "debug/u300.S" if DEBUG_U300_UART
default "debug/ux500.S" if DEBUG_UX500_UART
- default "debug/vexpress.S" if DEBUG_VEXPRESS_UART0_DETECT || \
- DEBUG_VEXPRESS_UART0_CA9 || DEBUG_VEXPRESS_UART0_RS1 || \
- DEBUG_VEXPRESS_UART0_CRX
+ default "debug/vexpress.S" if DEBUG_VEXPRESS_UART0_DETECT
default "debug/vt8500.S" if DEBUG_VT8500_UART0
default "debug/zynq.S" if DEBUG_ZYNQ_UART0 || DEBUG_ZYNQ_UART1
default "mach/debug-macro.S"
+# Compatibility options for PL01x
+config DEBUG_UART_PL01X
+ def_bool ARCH_EP93XX || \
+ ARCH_INTEGRATOR || \
+ ARCH_SPEAR3XX || \
+ ARCH_SPEAR6XX || \
+ ARCH_SPEAR13XX || \
+ ARCH_VERSATILE
+
+# Compatibility options for 8250
+config DEBUG_UART_8250
+ def_bool ARCH_DOVE || ARCH_EBSA110 || \
+ (FOOTBRIDGE && !DEBUG_DC21285_PORT) || \
+ ARCH_GEMINI || ARCH_IOP13XX || ARCH_IOP32X || \
+ ARCH_IOP33X || ARCH_IXP4XX || ARCH_KIRKWOOD || \
+ ARCH_LPC32XX || ARCH_MV78XX0 || ARCH_ORION5X || ARCH_RPC
+
+config DEBUG_UART_PHYS
+ hex "Physical base address of debug UART"
+ default 0x01c20000 if DEBUG_DAVINCI_DMx_UART0
+ default 0x01c28000 if DEBUG_SUNXI_UART0
+ default 0x01c28400 if DEBUG_SUNXI_UART1
+ default 0x01d0c000 if DEBUG_DAVINCI_DA8XX_UART1
+ default 0x01d0d000 if DEBUG_DAVINCI_DA8XX_UART2
+ default 0x02530c00 if DEBUG_KEYSTONE_UART0
+ default 0x02531000 if DEBUG_KEYSTONE_UART1
+ default 0x03010fe0 if ARCH_RPC
+ default 0x08108300 if DEBUG_DAVINCI_TNETV107X_UART1
+ default 0x10009000 if DEBUG_REALVIEW_STD_PORT || DEBUG_CNS3XXX || \
+ DEBUG_VEXPRESS_UART0_CA9
+ default 0x1010c000 if DEBUG_REALVIEW_PB1176_PORT
+ default 0x10124000 if DEBUG_RK3X_UART0
+ default 0x10126000 if DEBUG_RK3X_UART1
+ default 0x101f1000 if ARCH_VERSATILE
+ default 0x101fb000 if DEBUG_NOMADIK_UART
+ default 0x16000000 if ARCH_INTEGRATOR
+ default 0x1c090000 if DEBUG_VEXPRESS_UART0_RS1
+ default 0x20060000 if DEBUG_RK29_UART0
+ default 0x20064000 if DEBUG_RK29_UART1 || DEBUG_RK3X_UART2
+ default 0x20068000 if DEBUG_RK29_UART2 || DEBUG_RK3X_UART3
+ default 0x20201000 if DEBUG_BCM2835
+ default 0x40090000 if ARCH_LPC32XX
+ default 0x40100000 if DEBUG_PXA_UART1
+ default 0x42000000 if ARCH_GEMINI
+ default 0x7c0003f8 if FOOTBRIDGE
+ default 0x80230000 if DEBUG_PICOXCELL_UART
+ default 0x80070000 if DEBUG_IMX23_UART
+ default 0x80074000 if DEBUG_IMX28_UART
+ default 0x808c0000 if ARCH_EP93XX
+ default 0x90020000 if DEBUG_NSPIRE_CLASSIC_UART || DEBUG_NSPIRE_CX_UART
+ default 0xb0090000 if DEBUG_VEXPRESS_UART0_CRX
+ default 0xc0013000 if DEBUG_U300_UART
+ default 0xc8000000 if ARCH_IXP4XX && !CPU_BIG_ENDIAN
+ default 0xc8000003 if ARCH_IXP4XX && CPU_BIG_ENDIAN
+ default 0xd0000000 if ARCH_SPEAR3XX || ARCH_SPEAR6XX
+ default 0xd0012000 if DEBUG_MVEBU_UART
+ default 0xd4017000 if DEBUG_MMP_UART2
+ default 0xd4018000 if DEBUG_MMP_UART3
+ default 0xe0000000 if ARCH_SPEAR13XX
+ default 0xf0000be0 if ARCH_EBSA110
+ default 0xf1012000 if DEBUG_MVEBU_UART_ALTERNATE
+ default 0xf1012000 if ARCH_DOVE || ARCH_KIRKWOOD || ARCH_MV78XX0 || \
+ ARCH_ORION5X
+ default 0xf8b00000 if DEBUG_HI3716_UART
+ default 0xfcb00000 if DEBUG_HI3620_UART
+ default 0xfe800000 if ARCH_IOP32X
+ default 0xffc02000 if DEBUG_SOCFPGA_UART
+ default 0xffd82340 if ARCH_IOP13XX
+ default 0xfff36000 if DEBUG_HIGHBANK_UART
+ default 0xfffff700 if ARCH_IOP33X
+ depends on DEBUG_LL_UART_8250 || DEBUG_LL_UART_PL01X || \
+ DEBUG_UART_8250 || DEBUG_UART_PL01X
+
+config DEBUG_UART_VIRT
+ hex "Virtual base address of debug UART"
+ default 0xe0010fe0 if ARCH_RPC
+ default 0xf0000be0 if ARCH_EBSA110
+ default 0xf0009000 if DEBUG_CNS3XXX
+ default 0xf01fb000 if DEBUG_NOMADIK_UART
+ default 0xf0201000 if DEBUG_BCM2835
+ default 0xf11f1000 if ARCH_VERSATILE
+ default 0xf1600000 if ARCH_INTEGRATOR
+ default 0xf1c28000 if DEBUG_SUNXI_UART0
+ default 0xf1c28400 if DEBUG_SUNXI_UART1
+ default 0xf2100000 if DEBUG_PXA_UART1
+ default 0xf4090000 if ARCH_LPC32XX
+ default 0xf4200000 if ARCH_GEMINI
+ default 0xf8009000 if DEBUG_VEXPRESS_UART0_CA9
+ default 0xf8090000 if DEBUG_VEXPRESS_UART0_RS1
+ default 0xfb009000 if DEBUG_REALVIEW_STD_PORT
+ default 0xfb10c000 if DEBUG_REALVIEW_PB1176_PORT
+ default 0xfd000000 if ARCH_SPEAR3XX || ARCH_SPEAR6XX
+ default 0xfd000000 if ARCH_SPEAR13XX
+ default 0xfd012000 if ARCH_MV78XX0
+ default 0xfde12000 if ARCH_DOVE
+ default 0xfe012000 if ARCH_ORION5X
+ default 0xfe017000 if DEBUG_MMP_UART2
+ default 0xfe018000 if DEBUG_MMP_UART3
+ default 0xfe100000 if DEBUG_IMX23_UART || DEBUG_IMX28_UART
+ default 0xfe230000 if DEBUG_PICOXCELL_UART
+ default 0xfe800000 if ARCH_IOP32X
+ default 0xfeb00000 if DEBUG_HI3620_UART || DEBUG_HI3716_UART
+ default 0xfeb24000 if DEBUG_RK3X_UART0
+ default 0xfeb26000 if DEBUG_RK3X_UART1
+ default 0xfeb30c00 if DEBUG_KEYSTONE_UART0
+ default 0xfeb31000 if DEBUG_KEYSTONE_UART1
+ default 0xfec12000 if DEBUG_MVEBU_UART || DEBUG_MVEBU_UART_ALTERNATE
+ default 0xfed60000 if DEBUG_RK29_UART0
+ default 0xfed64000 if DEBUG_RK29_UART1 || DEBUG_RK3X_UART2
+ default 0xfed68000 if DEBUG_RK29_UART2 || DEBUG_RK3X_UART3
+ default 0xfec02000 if DEBUG_SOCFPGA_UART
+ default 0xfec20000 if DEBUG_DAVINCI_DMx_UART0
+ default 0xfed0c000 if DEBUG_DAVINCI_DA8XX_UART1
+ default 0xfed0d000 if DEBUG_DAVINCI_DA8XX_UART2
+ default 0xfed12000 if ARCH_KIRKWOOD
+ default 0xfedc0000 if ARCH_EP93XX
+ default 0xfee003f8 if FOOTBRIDGE
+ default 0xfee08300 if DEBUG_DAVINCI_TNETV107X_UART1
+ default 0xfee20000 if DEBUG_NSPIRE_CLASSIC_UART || DEBUG_NSPIRE_CX_UART
+ default 0xfef36000 if DEBUG_HIGHBANK_UART
+ default 0xfee82340 if ARCH_IOP13XX
+ default 0xfef00000 if ARCH_IXP4XX && !CPU_BIG_ENDIAN
+ default 0xfef00003 if ARCH_IXP4XX && CPU_BIG_ENDIAN
+ default 0xfefff700 if ARCH_IOP33X
+ default 0xff003000 if DEBUG_U300_UART
+ default DEBUG_UART_PHYS if !MMU
+ depends on DEBUG_LL_UART_8250 || DEBUG_LL_UART_PL01X || \
+ DEBUG_UART_8250 || DEBUG_UART_PL01X
+
+config DEBUG_UART_8250_SHIFT
+ int "Register offset shift for the 8250 debug UART"
+ depends on DEBUG_LL_UART_8250 || DEBUG_UART_8250
+ default 0 if FOOTBRIDGE || ARCH_IOP32X
+ default 2
+
+config DEBUG_UART_8250_WORD
+ bool "Use 32-bit accesses for 8250 UART"
+ depends on DEBUG_LL_UART_8250 || DEBUG_UART_8250
+ depends on DEBUG_UART_8250_SHIFT >= 2
+ default y if DEBUG_PICOXCELL_UART || DEBUG_SOCFPGA_UART || \
+ ARCH_KEYSTONE || \
+ DEBUG_DAVINCI_DMx_UART0 || DEBUG_DAVINCI_DA8XX_UART1 || \
+ DEBUG_DAVINCI_DA8XX_UART2 || DEBUG_DAVINCI_TNETV107X_UART1
+
+config DEBUG_UART_8250_FLOW_CONTROL
+ bool "Enable flow control for 8250 UART"
+ depends on DEBUG_LL_UART_8250 || DEBUG_UART_8250
+ default y if ARCH_EBSA110 || FOOTBRIDGE || ARCH_GEMINI || ARCH_RPC
+
config DEBUG_UNCOMPRESS
bool
depends on ARCH_MULTIPLATFORM
default y if DEBUG_LL && !DEBUG_OMAP2PLUS_UART && \
- !DEBUG_TEGRA_UART
+ (!DEBUG_TEGRA_UART || !ZBOOT_ROM)
help
This option influences the normal decompressor output for
multiplatform kernels. Normally, multiplatform kernels disable
mov r0, #INBOUND_NOT_COMING_UP
strb r0, [r8, #MCPM_SYNC_CLUSTER_INBOUND]
- dsb
+ dsb st
sev
mov r0, r11
dmb
mov \rscratch, #0
strb \rscratch, [\rbase, \rcpu]
- dsb
+ dsb st
sev
.endm
dmb
mov r1, #VLOCK_OWNER_NONE
strb r1, [r0, #VLOCK_OWNER_OFFSET]
- dsb
+ dsb st
sev
bx lr
ENDPROC(vlock_unlock)
#ifdef CONFIG_SMP
#if __LINUX_ARM_ARCH__ >= 7
.ifeqs "\mode","arm"
- ALT_SMP(dmb)
+ ALT_SMP(dmb ish)
.else
- ALT_SMP(W(dmb))
+ ALT_SMP(W(dmb) ish)
.endif
#elif __LINUX_ARM_ARCH__ == 6
ALT_SMP(mcr p15, 0, r0, c7, c10, 5) @ dmb
#endif
#if __LINUX_ARM_ARCH__ >= 7
-#define isb() __asm__ __volatile__ ("isb" : : : "memory")
-#define dsb() __asm__ __volatile__ ("dsb" : : : "memory")
-#define dmb() __asm__ __volatile__ ("dmb" : : : "memory")
+#define isb(option) __asm__ __volatile__ ("isb " #option : : : "memory")
+#define dsb(option) __asm__ __volatile__ ("dsb " #option : : : "memory")
+#define dmb(option) __asm__ __volatile__ ("dmb " #option : : : "memory")
#elif defined(CONFIG_CPU_XSC3) || __LINUX_ARM_ARCH__ == 6
-#define isb() __asm__ __volatile__ ("mcr p15, 0, %0, c7, c5, 4" \
+#define isb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c5, 4" \
: : "r" (0) : "memory")
-#define dsb() __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 4" \
+#define dsb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 4" \
: : "r" (0) : "memory")
-#define dmb() __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 5" \
+#define dmb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 5" \
: : "r" (0) : "memory")
#elif defined(CONFIG_CPU_FA526)
-#define isb() __asm__ __volatile__ ("mcr p15, 0, %0, c7, c5, 4" \
+#define isb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c5, 4" \
: : "r" (0) : "memory")
-#define dsb() __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 4" \
+#define dsb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 4" \
: : "r" (0) : "memory")
-#define dmb() __asm__ __volatile__ ("" : : : "memory")
+#define dmb(x) __asm__ __volatile__ ("" : : : "memory")
#else
-#define isb() __asm__ __volatile__ ("" : : : "memory")
-#define dsb() __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 4" \
+#define isb(x) __asm__ __volatile__ ("" : : : "memory")
+#define dsb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 4" \
: : "r" (0) : "memory")
-#define dmb() __asm__ __volatile__ ("" : : : "memory")
+#define dmb(x) __asm__ __volatile__ ("" : : : "memory")
#endif
#ifdef CONFIG_ARCH_HAS_BARRIERS
#elif defined(CONFIG_ARM_DMA_MEM_BUFFERABLE) || defined(CONFIG_SMP)
#define mb() do { dsb(); outer_sync(); } while (0)
#define rmb() dsb()
-#define wmb() mb()
+#define wmb() do { dsb(st); outer_sync(); } while (0)
#else
#define mb() barrier()
#define rmb() barrier()
#define smp_rmb() barrier()
#define smp_wmb() barrier()
#else
-#define smp_mb() dmb()
-#define smp_rmb() dmb()
-#define smp_wmb() dmb()
+#define smp_mb() dmb(ish)
+#define smp_rmb() smp_mb()
+#define smp_wmb() dmb(ishst)
#endif
#define read_barrier_depends() do { } while(0)
* Harvard caches are synchronised for the user space address range.
* This is used for the ARM private sys_cacheflush system call.
*/
-#define flush_cache_user_range(start,end) \
- __cpuc_coherent_user_range((start) & PAGE_MASK, PAGE_ALIGN(end))
+#define flush_cache_user_range(s,e) __cpuc_coherent_user_range(s,e)
/*
* Perform necessary cache operations to ensure that data previously
* set_pte_at() called from vmap_pte_range() does not
* have a DSB after cleaning the cache line.
*/
- dsb();
+ dsb(ishst);
}
static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
+++ /dev/null
-/*
- * arch/arm/include/asm/hardware/debug-8250.S
- *
- * Copyright (C) 1994-1999 Russell King
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <linux/serial_reg.h>
-
- .macro senduart,rd,rx
- strb \rd, [\rx, #UART_TX << UART_SHIFT]
- .endm
-
- .macro busyuart,rd,rx
-1002: ldrb \rd, [\rx, #UART_LSR << UART_SHIFT]
- and \rd, \rd, #UART_LSR_TEMT | UART_LSR_THRE
- teq \rd, #UART_LSR_TEMT | UART_LSR_THRE
- bne 1002b
- .endm
-
- .macro waituart,rd,rx
-#ifdef FLOW_CONTROL
-1001: ldrb \rd, [\rx, #UART_MSR << UART_SHIFT]
- tst \rd, #UART_MSR_CTS
- beq 1001b
-#endif
- .endm
+++ /dev/null
-/* arch/arm/include/asm/hardware/debug-pl01x.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
-*/
-#include <linux/amba/serial.h>
-
- .macro senduart,rd,rx
- strb \rd, [\rx, #UART01x_DR]
- .endm
-
- .macro waituart,rd,rx
-1001: ldr \rd, [\rx, #UART01x_FR]
- tst \rd, #UART01x_FR_TXFF
- bne 1001b
- .endm
-
- .macro busyuart,rd,rx
-1001: ldr \rd, [\rx, #UART01x_FR]
- tst \rd, #UART01x_FR_BUSY
- bne 1001b
- .endm
/*
* Current machine - only accessible during boot.
*/
-extern struct machine_desc *machine_desc;
+extern const struct machine_desc *machine_desc;
/*
* Machine type table - also only accessible during boot
*/
-extern struct machine_desc __arch_info_begin[], __arch_info_end[];
+extern const struct machine_desc __arch_info_begin[], __arch_info_end[];
#define for_each_machine_desc(p) \
for (p = __arch_info_begin; p < __arch_info_end; p++)
struct meminfo;
struct machine_desc;
-extern void arm_memblock_init(struct meminfo *, struct machine_desc *);
-
+void arm_memblock_init(struct meminfo *, const struct machine_desc *);
phys_addr_t arm_memblock_steal(phys_addr_t size, phys_addr_t align);
#endif
ARM_SEC_CORE,
ARM_SEC_EXIT,
ARM_SEC_DEVEXIT,
+ ARM_SEC_HOT,
+ ARM_SEC_UNLIKELY,
ARM_SEC_MAX,
};
--- /dev/null
+/*
+ * linux/arch/arm/include/asm/neon.h
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <asm/hwcap.h>
+
+#define cpu_has_neon() (!!(elf_hwcap & HWCAP_NEON))
+
+#ifdef __ARM_NEON__
+
+/*
+ * If you are affected by the BUILD_BUG below, it probably means that you are
+ * using NEON code /and/ calling the kernel_neon_begin() function from the same
+ * compilation unit. To prevent issues that may arise from GCC reordering or
+ * generating(1) NEON instructions outside of these begin/end functions, the
+ * only supported way of using NEON code in the kernel is by isolating it in a
+ * separate compilation unit, and calling it from another unit from inside a
+ * kernel_neon_begin/kernel_neon_end pair.
+ *
+ * (1) Current GCC (4.7) might generate NEON instructions at O3 level if
+ * -mpfu=neon is set.
+ */
+
+#define kernel_neon_begin() \
+ BUILD_BUG_ON_MSG(1, "kernel_neon_begin() called from NEON code")
+
+#else
+void kernel_neon_begin(void);
+#endif
+void kernel_neon_end(void);
#define PAGE_HYP _MOD_PROT(pgprot_kernel, L_PTE_HYP)
#define PAGE_HYP_DEVICE _MOD_PROT(pgprot_hyp_device, L_PTE_HYP)
#define PAGE_S2 _MOD_PROT(pgprot_s2, L_PTE_S2_RDONLY)
-#define PAGE_S2_DEVICE _MOD_PROT(pgprot_s2_device, L_PTE_USER | L_PTE_S2_RDONLY)
+#define PAGE_S2_DEVICE _MOD_PROT(pgprot_s2_device, L_PTE_S2_RDWR)
#define __PAGE_NONE __pgprot(_L_PTE_DEFAULT | L_PTE_RDONLY | L_PTE_XN | L_PTE_NONE)
#define __PAGE_SHARED __pgprot(_L_PTE_DEFAULT | L_PTE_USER | L_PTE_XN)
#ifdef CONFIG_OF
-extern struct machine_desc *setup_machine_fdt(unsigned int dt_phys);
+extern const struct machine_desc *setup_machine_fdt(unsigned int dt_phys);
extern void arm_dt_memblock_reserve(void);
extern void __init arm_dt_init_cpu_maps(void);
#else /* CONFIG_OF */
-static inline struct machine_desc *setup_machine_fdt(unsigned int dt_phys)
+static inline const struct machine_desc *setup_machine_fdt(unsigned int dt_phys)
{
return NULL;
}
{
#if __LINUX_ARM_ARCH__ >= 7
__asm__ __volatile__ (
- "dsb\n"
+ "dsb ishst\n"
SEV
);
#else
#include <linux/thread_info.h>
+/*
+ * For v7 SMP cores running a preemptible kernel we may be pre-empted
+ * during a TLB maintenance operation, so execute an inner-shareable dsb
+ * to ensure that the maintenance completes in case we migrate to another
+ * CPU.
+ */
+#if defined(CONFIG_PREEMPT) && defined(CONFIG_SMP) && defined(CONFIG_CPU_V7)
+#define finish_arch_switch(prev) dsb(ish)
+#endif
+
/*
* switch_to(prev, next) should switch from task `prev' to `next'
* `prev' will never be the same as `next'. schedule() itself
__u32 extra[2]; /* Xscale 'acc' register, etc */
};
+struct arm_restart_block {
+ union {
+ /* For user cache flushing */
+ struct {
+ unsigned long start;
+ unsigned long end;
+ } cache;
+ };
+};
+
/*
* low level task data that entry.S needs immediate access to.
* __switch_to() assumes cpu_context follows immediately after cpu_domain.
unsigned long thumbee_state; /* ThumbEE Handler Base register */
#endif
struct restart_block restart_block;
+ struct arm_restart_block arm_restart_block;
};
#define INIT_THREAD_INFO(tsk) \
#define tlb_op(f, regs, arg) __tlb_op(f, "p15, 0, %0, " regs, arg)
#define tlb_l2_op(f, regs, arg) __tlb_op(f, "p15, 1, %0, " regs, arg)
-static inline void local_flush_tlb_all(void)
+static inline void __local_flush_tlb_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = __cpu_tlb_flags;
- if (tlb_flag(TLB_WB))
- dsb();
-
tlb_op(TLB_V4_U_FULL | TLB_V6_U_FULL, "c8, c7, 0", zero);
tlb_op(TLB_V4_D_FULL | TLB_V6_D_FULL, "c8, c6, 0", zero);
tlb_op(TLB_V4_I_FULL | TLB_V6_I_FULL, "c8, c5, 0", zero);
- tlb_op(TLB_V7_UIS_FULL, "c8, c3, 0", zero);
+}
+
+static inline void local_flush_tlb_all(void)
+{
+ const int zero = 0;
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ if (tlb_flag(TLB_WB))
+ dsb(nshst);
+
+ __local_flush_tlb_all();
+ tlb_op(TLB_V7_UIS_FULL, "c8, c7, 0", zero);
if (tlb_flag(TLB_BARRIER)) {
- dsb();
+ dsb(nsh);
isb();
}
}
-static inline void local_flush_tlb_mm(struct mm_struct *mm)
+static inline void __flush_tlb_all(void)
{
const int zero = 0;
- const int asid = ASID(mm);
const unsigned int __tlb_flag = __cpu_tlb_flags;
if (tlb_flag(TLB_WB))
- dsb();
+ dsb(ishst);
+
+ __local_flush_tlb_all();
+ tlb_op(TLB_V7_UIS_FULL, "c8, c3, 0", zero);
+
+ if (tlb_flag(TLB_BARRIER)) {
+ dsb(ish);
+ isb();
+ }
+}
+
+static inline void __local_flush_tlb_mm(struct mm_struct *mm)
+{
+ const int zero = 0;
+ const int asid = ASID(mm);
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
if (possible_tlb_flags & (TLB_V4_U_FULL|TLB_V4_D_FULL|TLB_V4_I_FULL)) {
- if (cpumask_test_cpu(get_cpu(), mm_cpumask(mm))) {
+ if (cpumask_test_cpu(smp_processor_id(), mm_cpumask(mm))) {
tlb_op(TLB_V4_U_FULL, "c8, c7, 0", zero);
tlb_op(TLB_V4_D_FULL, "c8, c6, 0", zero);
tlb_op(TLB_V4_I_FULL, "c8, c5, 0", zero);
}
- put_cpu();
}
tlb_op(TLB_V6_U_ASID, "c8, c7, 2", asid);
tlb_op(TLB_V6_D_ASID, "c8, c6, 2", asid);
tlb_op(TLB_V6_I_ASID, "c8, c5, 2", asid);
+}
+
+static inline void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ const int asid = ASID(mm);
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ if (tlb_flag(TLB_WB))
+ dsb(nshst);
+
+ __local_flush_tlb_mm(mm);
+ tlb_op(TLB_V7_UIS_ASID, "c8, c7, 2", asid);
+
+ if (tlb_flag(TLB_BARRIER))
+ dsb(nsh);
+}
+
+static inline void __flush_tlb_mm(struct mm_struct *mm)
+{
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ if (tlb_flag(TLB_WB))
+ dsb(ishst);
+
+ __local_flush_tlb_mm(mm);
#ifdef CONFIG_ARM_ERRATA_720789
- tlb_op(TLB_V7_UIS_ASID, "c8, c3, 0", zero);
+ tlb_op(TLB_V7_UIS_ASID, "c8, c3, 0", 0);
#else
- tlb_op(TLB_V7_UIS_ASID, "c8, c3, 2", asid);
+ tlb_op(TLB_V7_UIS_ASID, "c8, c3, 2", ASID(mm));
#endif
if (tlb_flag(TLB_BARRIER))
- dsb();
+ dsb(ish);
}
static inline void
-local_flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
+__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
{
const int zero = 0;
const unsigned int __tlb_flag = __cpu_tlb_flags;
uaddr = (uaddr & PAGE_MASK) | ASID(vma->vm_mm);
- if (tlb_flag(TLB_WB))
- dsb();
-
if (possible_tlb_flags & (TLB_V4_U_PAGE|TLB_V4_D_PAGE|TLB_V4_I_PAGE|TLB_V4_I_FULL) &&
cpumask_test_cpu(smp_processor_id(), mm_cpumask(vma->vm_mm))) {
tlb_op(TLB_V4_U_PAGE, "c8, c7, 1", uaddr);
tlb_op(TLB_V6_U_PAGE, "c8, c7, 1", uaddr);
tlb_op(TLB_V6_D_PAGE, "c8, c6, 1", uaddr);
tlb_op(TLB_V6_I_PAGE, "c8, c5, 1", uaddr);
+}
+
+static inline void
+local_flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
+{
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ uaddr = (uaddr & PAGE_MASK) | ASID(vma->vm_mm);
+
+ if (tlb_flag(TLB_WB))
+ dsb(nshst);
+
+ __local_flush_tlb_page(vma, uaddr);
+ tlb_op(TLB_V7_UIS_PAGE, "c8, c7, 1", uaddr);
+
+ if (tlb_flag(TLB_BARRIER))
+ dsb(nsh);
+}
+
+static inline void
+__flush_tlb_page(struct vm_area_struct *vma, unsigned long uaddr)
+{
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ uaddr = (uaddr & PAGE_MASK) | ASID(vma->vm_mm);
+
+ if (tlb_flag(TLB_WB))
+ dsb(ishst);
+
+ __local_flush_tlb_page(vma, uaddr);
#ifdef CONFIG_ARM_ERRATA_720789
tlb_op(TLB_V7_UIS_PAGE, "c8, c3, 3", uaddr & PAGE_MASK);
#else
#endif
if (tlb_flag(TLB_BARRIER))
- dsb();
+ dsb(ish);
}
-static inline void local_flush_tlb_kernel_page(unsigned long kaddr)
+static inline void __local_flush_tlb_kernel_page(unsigned long kaddr)
{
const int zero = 0;
const unsigned int __tlb_flag = __cpu_tlb_flags;
- kaddr &= PAGE_MASK;
-
- if (tlb_flag(TLB_WB))
- dsb();
-
tlb_op(TLB_V4_U_PAGE, "c8, c7, 1", kaddr);
tlb_op(TLB_V4_D_PAGE, "c8, c6, 1", kaddr);
tlb_op(TLB_V4_I_PAGE, "c8, c5, 1", kaddr);
tlb_op(TLB_V6_U_PAGE, "c8, c7, 1", kaddr);
tlb_op(TLB_V6_D_PAGE, "c8, c6, 1", kaddr);
tlb_op(TLB_V6_I_PAGE, "c8, c5, 1", kaddr);
+}
+
+static inline void local_flush_tlb_kernel_page(unsigned long kaddr)
+{
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ kaddr &= PAGE_MASK;
+
+ if (tlb_flag(TLB_WB))
+ dsb(nshst);
+
+ __local_flush_tlb_kernel_page(kaddr);
+ tlb_op(TLB_V7_UIS_PAGE, "c8, c7, 1", kaddr);
+
+ if (tlb_flag(TLB_BARRIER)) {
+ dsb(nsh);
+ isb();
+ }
+}
+
+static inline void __flush_tlb_kernel_page(unsigned long kaddr)
+{
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ kaddr &= PAGE_MASK;
+
+ if (tlb_flag(TLB_WB))
+ dsb(ishst);
+
+ __local_flush_tlb_kernel_page(kaddr);
tlb_op(TLB_V7_UIS_PAGE, "c8, c3, 1", kaddr);
if (tlb_flag(TLB_BARRIER)) {
- dsb();
+ dsb(ish);
isb();
}
}
+/*
+ * Branch predictor maintenance is paired with full TLB invalidation, so
+ * there is no need for any barriers here.
+ */
+static inline void __local_flush_bp_all(void)
+{
+ const int zero = 0;
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ if (tlb_flag(TLB_V6_BP))
+ asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero));
+}
+
static inline void local_flush_bp_all(void)
{
const int zero = 0;
const unsigned int __tlb_flag = __cpu_tlb_flags;
+ __local_flush_bp_all();
if (tlb_flag(TLB_V7_UIS_BP))
- asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero));
- else if (tlb_flag(TLB_V6_BP))
asm("mcr p15, 0, %0, c7, c5, 6" : : "r" (zero));
+}
- if (tlb_flag(TLB_BARRIER))
- isb();
+static inline void __flush_bp_all(void)
+{
+ const int zero = 0;
+ const unsigned int __tlb_flag = __cpu_tlb_flags;
+
+ __local_flush_bp_all();
+ if (tlb_flag(TLB_V7_UIS_BP))
+ asm("mcr p15, 0, %0, c7, c1, 6" : : "r" (zero));
}
#include <asm/cputype.h>
* Dummy TLBIMVAIS. Using the unmapped address 0 and ASID 0.
*/
asm("mcr p15, 0, %0, c8, c3, 1" : : "r" (0));
- dsb();
+ dsb(ish);
}
#else
static inline int erratum_a15_798181(void)
tlb_l2_op(TLB_L2CLEAN_FR, "c15, c9, 1 @ L2 flush_pmd", pmd);
if (tlb_flag(TLB_WB))
- dsb();
+ dsb(ishst);
}
static inline void clean_pmd_entry(void *pmd)
--- /dev/null
+#ifndef _ASM_TYPES_H
+#define _ASM_TYPES_H
+
+#include <asm-generic/int-ll64.h>
+
+/*
+ * The C99 types uintXX_t that are usually defined in 'stdint.h' are not as
+ * unambiguous on ARM as you would expect. For the types below, there is a
+ * difference on ARM between GCC built for bare metal ARM, GCC built for glibc
+ * and the kernel itself, which results in build errors if you try to build with
+ * -ffreestanding and include 'stdint.h' (such as when you include 'arm_neon.h'
+ * in order to use NEON intrinsics)
+ *
+ * As the typedefs for these types in 'stdint.h' are based on builtin defines
+ * supplied by GCC, we can tweak these to align with the kernel's idea of those
+ * types, so 'linux/types.h' and 'stdint.h' can be safely included from the same
+ * source file (provided that -ffreestanding is used).
+ *
+ * int32_t uint32_t uintptr_t
+ * bare metal GCC long unsigned long unsigned int
+ * glibc GCC int unsigned int unsigned int
+ * kernel int unsigned int unsigned long
+ */
+
+#ifdef __INT32_TYPE__
+#undef __INT32_TYPE__
+#define __INT32_TYPE__ int
+#endif
+
+#ifdef __UINT32_TYPE__
+#undef __UINT32_TYPE__
+#define __UINT32_TYPE__ unsigned int
+#endif
+
+#ifdef __UINTPTR_TYPE__
+#undef __UINTPTR_TYPE__
+#define __UINTPTR_TYPE__ unsigned long
+#endif
+
+#endif /* _ASM_TYPES_H */
#define V7M_SCB_VTOR 0x08
+#define V7M_SCB_AIRCR 0x0c
+#define V7M_SCB_AIRCR_VECTKEY (0x05fa << 16)
+#define V7M_SCB_AIRCR_SYSRESETREQ (1 << 2)
+
#define V7M_SCB_SCR 0x10
#define V7M_SCB_SCR_SLEEPDEEP (1 << 2)
*/
#define EXC_RET_STACK_MASK 0x00000004
#define EXC_RET_THREADMODE_PROCESSSTACK 0xfffffffd
+
+#ifndef __ASSEMBLY__
+
+enum reboot_mode;
+
+void armv7m_restart(enum reboot_mode mode, const char *cmd);
+
+#endif /* __ASSEMBLY__ */
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
+#include <linux/hardirq.h>
#include <asm-generic/xor.h>
+#include <asm/hwcap.h>
+#include <asm/neon.h>
#define __XOR(a1, a2) a1 ^= a2
xor_speed(&xor_block_arm4regs); \
xor_speed(&xor_block_8regs); \
xor_speed(&xor_block_32regs); \
+ NEON_TEMPLATES; \
} while (0)
+
+#ifdef CONFIG_KERNEL_MODE_NEON
+
+extern struct xor_block_template const xor_block_neon_inner;
+
+static void
+xor_neon_2(unsigned long bytes, unsigned long *p1, unsigned long *p2)
+{
+ if (in_interrupt()) {
+ xor_arm4regs_2(bytes, p1, p2);
+ } else {
+ kernel_neon_begin();
+ xor_block_neon_inner.do_2(bytes, p1, p2);
+ kernel_neon_end();
+ }
+}
+
+static void
+xor_neon_3(unsigned long bytes, unsigned long *p1, unsigned long *p2,
+ unsigned long *p3)
+{
+ if (in_interrupt()) {
+ xor_arm4regs_3(bytes, p1, p2, p3);
+ } else {
+ kernel_neon_begin();
+ xor_block_neon_inner.do_3(bytes, p1, p2, p3);
+ kernel_neon_end();
+ }
+}
+
+static void
+xor_neon_4(unsigned long bytes, unsigned long *p1, unsigned long *p2,
+ unsigned long *p3, unsigned long *p4)
+{
+ if (in_interrupt()) {
+ xor_arm4regs_4(bytes, p1, p2, p3, p4);
+ } else {
+ kernel_neon_begin();
+ xor_block_neon_inner.do_4(bytes, p1, p2, p3, p4);
+ kernel_neon_end();
+ }
+}
+
+static void
+xor_neon_5(unsigned long bytes, unsigned long *p1, unsigned long *p2,
+ unsigned long *p3, unsigned long *p4, unsigned long *p5)
+{
+ if (in_interrupt()) {
+ xor_arm4regs_5(bytes, p1, p2, p3, p4, p5);
+ } else {
+ kernel_neon_begin();
+ xor_block_neon_inner.do_5(bytes, p1, p2, p3, p4, p5);
+ kernel_neon_end();
+ }
+}
+
+static struct xor_block_template xor_block_neon = {
+ .name = "neon",
+ .do_2 = xor_neon_2,
+ .do_3 = xor_neon_3,
+ .do_4 = xor_neon_4,
+ .do_5 = xor_neon_5
+};
+
+#define NEON_TEMPLATES \
+ do { if (cpu_has_neon()) xor_speed(&xor_block_neon); } while (0)
+#else
+#define NEON_TEMPLATES
+#endif
--- /dev/null
+/*
+ * arch/arm/include/debug/8250.S
+ *
+ * Copyright (C) 1994-2013 Russell King
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/serial_reg.h>
+
+ .macro addruart, rp, rv, tmp
+ ldr \rp, =CONFIG_DEBUG_UART_PHYS
+ ldr \rv, =CONFIG_DEBUG_UART_VIRT
+ .endm
+
+#ifdef CONFIG_DEBUG_UART_8250_WORD
+ .macro store, rd, rx:vararg
+ str \rd, \rx
+ .endm
+
+ .macro load, rd, rx:vararg
+ ldr \rd, \rx
+ .endm
+#else
+ .macro store, rd, rx:vararg
+ strb \rd, \rx
+ .endm
+
+ .macro load, rd, rx:vararg
+ ldrb \rd, \rx
+ .endm
+#endif
+
+#define UART_SHIFT CONFIG_DEBUG_UART_8250_SHIFT
+
+ .macro senduart,rd,rx
+ store \rd, [\rx, #UART_TX << UART_SHIFT]
+ .endm
+
+ .macro busyuart,rd,rx
+1002: load \rd, [\rx, #UART_LSR << UART_SHIFT]
+ and \rd, \rd, #UART_LSR_TEMT | UART_LSR_THRE
+ teq \rd, #UART_LSR_TEMT | UART_LSR_THRE
+ bne 1002b
+ .endm
+
+ .macro waituart,rd,rx
+#ifdef CONFIG_DEBUG_UART_8250_FLOW_CONTROL
+1001: load \rd, [\rx, #UART_MSR << UART_SHIFT]
+ tst \rd, #UART_MSR_CTS
+ beq 1001b
+#endif
+ .endm
+++ /dev/null
-/*
- * Copyright (c) 2011 Picochip Ltd., Jamie Iles
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * Derived from arch/arm/mach-davinci/include/mach/debug-macro.S to use 32-bit
- * accesses to the 8250.
- */
-
-#include <linux/serial_reg.h>
-
- .macro senduart,rd,rx
- str \rd, [\rx, #UART_TX << UART_SHIFT]
- .endm
-
- .macro busyuart,rd,rx
-1002: ldr \rd, [\rx, #UART_LSR << UART_SHIFT]
- and \rd, \rd, #UART_LSR_TEMT | UART_LSR_THRE
- teq \rd, #UART_LSR_TEMT | UART_LSR_THRE
- bne 1002b
- .endm
-
- /* The UART's don't have any flow control IO's wired up. */
- .macro waituart,rd,rx
- .endm
+++ /dev/null
-/*
- * Debugging macro include header
- *
- * Copyright (C) 2010 Broadcom
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- */
-
-#define BCM2835_DEBUG_PHYS 0x20201000
-#define BCM2835_DEBUG_VIRT 0xf0201000
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =BCM2835_DEBUG_PHYS
- ldr \rv, =BCM2835_DEBUG_VIRT
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/*
- * Debugging macro include header
- *
- * Copyright 1994-1999 Russell King
- * Copyright 2008 Cavium Networks
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This file is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License, Version 2, as
- * published by the Free Software Foundation.
- */
-
- .macro addruart,rp,rv,tmp
- mov \rp, #0x00009000
- orr \rv, \rp, #0xf0000000 @ virtual base
- orr \rp, \rp, #0x10000000
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/*
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
- .macro addruart,rp,rv,tmp
- ldr \rv, =0xfee36000
- ldr \rp, =0xfff36000
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/*
- * Early serial debug output macro for Keystone SOCs
- *
- * Copyright 2013 Texas Instruments, Inc.
- * Santosh Shilimkar <santosh.shilimkar@ti.com>
- *
- * Based on RMKs low level debug code.
- * Copyright (C) 1994-1999 Russell King
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#include <linux/serial_reg.h>
-
-#define UART_SHIFT 2
-#if defined(CONFIG_DEBUG_KEYSTONE_UART0)
-#define UART_PHYS 0x02530c00
-#define UART_VIRT 0xfeb30c00
-#elif defined(CONFIG_DEBUG_KEYSTONE_UART1)
-#define UART_PHYS 0x02531000
-#define UART_VIRT 0xfeb31000
-#endif
-
- .macro addruart, rp, rv, tmp
- ldr \rv, =UART_VIRT @ physical base address
- ldr \rp, =UART_PHYS @ virtual base address
- .endm
-
- .macro senduart,rd,rx
- str \rd, [\rx, #UART_TX << UART_SHIFT]
- .endm
-
- .macro busyuart,rd,rx
-1002: ldr \rd, [\rx, #UART_LSR << UART_SHIFT]
- and \rd, \rd, #UART_LSR_TEMT | UART_LSR_THRE
- teq \rd, #UART_LSR_TEMT | UART_LSR_THRE
- bne 1002b
- .endm
-
- .macro waituart,rd,rx
- .endm
+++ /dev/null
-/*
- * Early serial output macro for Marvell SoC
- *
- * Copyright (C) 2012 Marvell
- *
- * Lior Amsalem <alior@marvell.com>
- * Gregory Clement <gregory.clement@free-electrons.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#ifdef CONFIG_DEBUG_MVEBU_UART_ALTERNATE
-#define ARMADA_370_XP_REGS_PHYS_BASE 0xf1000000
-#else
-#define ARMADA_370_XP_REGS_PHYS_BASE 0xd0000000
-#endif
-
-#define ARMADA_370_XP_REGS_VIRT_BASE 0xfec00000
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =ARMADA_370_XP_REGS_PHYS_BASE
- ldr \rv, =ARMADA_370_XP_REGS_VIRT_BASE
- orr \rp, \rp, #0x00012000
- orr \rv, \rv, #0x00012000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/* arch/arm/mach-mxs/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- */
-
-#ifdef CONFIG_DEBUG_IMX23_UART
-#define UART_PADDR 0x80070000
-#elif defined (CONFIG_DEBUG_IMX28_UART)
-#define UART_PADDR 0x80074000
-#endif
-
-#define UART_VADDR 0xfe100000
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =UART_PADDR @ physical
- ldr \rv, =UART_VADDR @ virtual
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/*
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
-*/
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0x00100000
- add \rp, \rp, #0x000fb000
- add \rv, \rp, #0xf0000000 @ virtual base
- add \rp, \rp, #0x10000000 @ physical base address
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/*
- * linux/arch/arm/include/debug/nspire.S
- *
- * Copyright (C) 2013 Daniel Tang <tangrs@tangrs.id.au>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2, as
- * published by the Free Software Foundation.
- *
- */
-
-#define NSPIRE_EARLY_UART_PHYS_BASE 0x90020000
-#define NSPIRE_EARLY_UART_VIRT_BASE 0xfee20000
-
-.macro addruart, rp, rv, tmp
- ldr \rp, =(NSPIRE_EARLY_UART_PHYS_BASE) @ physical base address
- ldr \rv, =(NSPIRE_EARLY_UART_VIRT_BASE) @ virtual base address
-.endm
-
-
-#ifdef CONFIG_DEBUG_NSPIRE_CX_UART
-#include <asm/hardware/debug-pl01x.S>
-#endif
-
-#ifdef CONFIG_DEBUG_NSPIRE_CLASSIC_UART
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
-#endif
+++ /dev/null
-/*
- * Copyright (c) 2011 Picochip Ltd., Jamie Iles
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- */
-
-#define UART_SHIFT 2
-#define PICOXCELL_UART1_BASE 0x80230000
-#define PHYS_TO_IO(x) (((x) & 0x00ffffff) | 0xfe000000)
-
- .macro addruart, rp, rv, tmp
- ldr \rv, =PHYS_TO_IO(PICOXCELL_UART1_BASE)
- ldr \rp, =PICOXCELL_UART1_BASE
- .endm
-
-#include "8250_32.S"
--- /dev/null
+/* arch/arm/include/debug/pl01x.S
+ *
+ * Debugging macro include header
+ *
+ * Copyright (C) 1994-1999 Russell King
+ * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+*/
+#include <linux/amba/serial.h>
+
+#ifdef CONFIG_DEBUG_UART_PHYS
+ .macro addruart, rp, rv, tmp
+ ldr \rp, =CONFIG_DEBUG_UART_PHYS
+ ldr \rv, =CONFIG_DEBUG_UART_VIRT
+ .endm
+#endif
+
+ .macro senduart,rd,rx
+ strb \rd, [\rx, #UART01x_DR]
+ .endm
+
+ .macro waituart,rd,rx
+1001: ldr \rd, [\rx, #UART01x_FR]
+ tst \rd, #UART01x_FR_TXFF
+ bne 1001b
+ .endm
+
+ .macro busyuart,rd,rx
+1001: ldr \rd, [\rx, #UART01x_FR]
+ tst \rd, #UART01x_FR_BUSY
+ bne 1001b
+ .endm
+++ /dev/null
-/*
- * Early serial output macro for Marvell PXA/MMP SoC
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * Copyright (C) 2013 Haojian Zhuang
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#if defined(CONFIG_DEBUG_PXA_UART1)
-#define PXA_UART_REG_PHYS_BASE 0x40100000
-#define PXA_UART_REG_VIRT_BASE 0xf2100000
-#elif defined(CONFIG_DEBUG_MMP_UART2)
-#define PXA_UART_REG_PHYS_BASE 0xd4017000
-#define PXA_UART_REG_VIRT_BASE 0xfe017000
-#elif defined(CONFIG_DEBUG_MMP_UART3)
-#define PXA_UART_REG_PHYS_BASE 0xd4018000
-#define PXA_UART_REG_VIRT_BASE 0xfe018000
-#else
-#error "Select uart for DEBUG_LL"
-#endif
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =PXA_UART_REG_PHYS_BASE
- ldr \rv, =PXA_UART_REG_VIRT_BASE
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * Early serial output macro for Rockchip SoCs
- *
- * Copyright (C) 2012 Maxime Ripard
- *
- * Maxime Ripard <maxime.ripard@free-electrons.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#if defined(CONFIG_DEBUG_RK29_UART0)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x20060000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfed60000
-#elif defined(CONFIG_DEBUG_RK29_UART1)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x20064000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfed64000
-#elif defined(CONFIG_DEBUG_RK29_UART2)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x20068000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfed68000
-#elif defined(CONFIG_DEBUG_RK3X_UART0)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x10124000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfeb24000
-#elif defined(CONFIG_DEBUG_RK3X_UART1)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x10126000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfeb26000
-#elif defined(CONFIG_DEBUG_RK3X_UART2)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x20064000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfed64000
-#elif defined(CONFIG_DEBUG_RK3X_UART3)
-#define ROCKCHIP_UART_DEBUG_PHYS_BASE 0x20068000
-#define ROCKCHIP_UART_DEBUG_VIRT_BASE 0xfed68000
-#endif
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =ROCKCHIP_UART_DEBUG_PHYS_BASE
- ldr \rv, =ROCKCHIP_UART_DEBUG_VIRT_BASE
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#define UART_SHIFT 2
-#define DEBUG_LL_UART_OFFSET 0x00002000
-
- .macro addruart, rp, rv, tmp
- mov \rp, #DEBUG_LL_UART_OFFSET
- orr \rp, \rp, #0x00c00000
- orr \rv, \rp, #0xfe000000 @ virtual base
- orr \rp, \rp, #0xff000000 @ physical base
- .endm
-
-#include "8250_32.S"
-
+++ /dev/null
-/*
- * Early serial output macro for Allwinner A1X SoCs
- *
- * Copyright (C) 2012 Maxime Ripard
- *
- * Maxime Ripard <maxime.ripard@free-electrons.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#if defined(CONFIG_DEBUG_SUNXI_UART0)
-#define SUNXI_UART_DEBUG_PHYS_BASE 0x01c28000
-#define SUNXI_UART_DEBUG_VIRT_BASE 0xf1c28000
-#elif defined(CONFIG_DEBUG_SUNXI_UART1)
-#define SUNXI_UART_DEBUG_PHYS_BASE 0x01c28400
-#define SUNXI_UART_DEBUG_VIRT_BASE 0xf1c28400
-#endif
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =SUNXI_UART_DEBUG_PHYS_BASE
- ldr \rv, =SUNXI_UART_DEBUG_VIRT_BASE
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
1002:
#endif
.endm
+
+/*
+ * Storage for the state maintained by the macros above.
+ *
+ * In the kernel proper, this data is located in arch/arm/mach-tegra/common.c.
+ * That's because this header is included from multiple files, and we only
+ * want a single copy of the data. In particular, the UART probing code above
+ * assumes it's running using physical addresses. This is true when this file
+ * is included from head.o, but not when included from debug.o. So we need
+ * to share the probe results between the two copies, rather than having
+ * to re-run the probing again later.
+ *
+ * In the decompressor, we put the symbol/storage right here, since common.c
+ * isn't included in the decompressor build. This symbol gets put in .text
+ * even though it's really data, since .data is discarded from the
+ * decompressor. Luckily, .text is writeable in the decompressor, unless
+ * CONFIG_ZBOOT_ROM. That dependency is handled in arch/arm/Kconfig.debug.
+ */
+#if defined(ZIMAGE)
+tegra_uart_config:
+ /* Debug UART initialization required */
+ .word 1
+ /* Debug UART physical address */
+ .word 0
+ /* Debug UART virtual address */
+ .word 0
+ /* Scratch space for debug macro */
+ .word 0
+#endif
+++ /dev/null
-/*
- * Copyright (C) 2006-2013 ST-Ericsson AB
- * License terms: GNU General Public License (GPL) version 2
- * Debugging macro include header.
- * Author: Linus Walleij <linus.walleij@stericsson.com>
- */
-#define U300_SLOW_PER_PHYS_BASE 0xc0010000
-#define U300_SLOW_PER_VIRT_BASE 0xff000000
-
- .macro addruart, rp, rv, tmp
- /* If we move the address using MMU, use this. */
- ldr \rp, = U300_SLOW_PER_PHYS_BASE @ MMU off, physical address
- ldr \rv, = U300_SLOW_PER_VIRT_BASE @ MMU on, virtual address
- orr \rp, \rp, #0x00003000
- orr \rv, \rv, #0x00003000
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
ldr \rv, =UART_VIRT_BASE @ yes, virtual address
.endm
-#include <asm/hardware/debug-pl01x.S>
+#include <debug/pl01x.S>
.endm
-#include <asm/hardware/debug-pl01x.S>
-
-#elif defined(CONFIG_DEBUG_VEXPRESS_UART0_CA9)
-
- .macro addruart,rp,rv,tmp
- mov \rp, #DEBUG_LL_UART_OFFSET
- orr \rv, \rp, #DEBUG_LL_VIRT_BASE
- orr \rp, \rp, #DEBUG_LL_PHYS_BASE
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
-
-#elif defined(CONFIG_DEBUG_VEXPRESS_UART0_RS1)
-
- .macro addruart,rp,rv,tmp
- mov \rp, #DEBUG_LL_UART_OFFSET_RS1
- orr \rv, \rp, #DEBUG_LL_VIRT_BASE
- orr \rp, \rp, #DEBUG_LL_PHYS_BASE_RS1
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
-
-#elif defined(CONFIG_DEBUG_VEXPRESS_UART0_CRX)
-
- .macro addruart,rp,tmp,tmp2
- ldr \rp, =DEBUG_LL_UART_PHYS_CRX
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
-
-#else /* CONFIG_DEBUG_LL_UART_NONE */
-
- .macro addruart, rp, rv, tmp
- /* Safe dummy values */
- mov \rp, #0
- mov \rv, #DEBUG_LL_VIRT_BASE
- .endm
-
- .macro senduart,rd,rx
- .endm
-
- .macro waituart,rd,rx
- .endm
-
- .macro busyuart,rd,rx
- .endm
-
+#include <debug/pl01x.S>
#endif
obj-$(CONFIG_DEPRECATED_PARAM_STRUCT) += atags_compat.o
ifeq ($(CONFIG_CPU_V7M),y)
-obj-y += entry-v7m.o
+obj-y += entry-v7m.o v7m.o
else
obj-y += entry-armv.o
endif
void convert_to_tag_list(struct tag *tags);
#ifdef CONFIG_ATAGS
-struct machine_desc *setup_machine_tags(phys_addr_t __atags_pointer, unsigned int machine_nr);
+const struct machine_desc *setup_machine_tags(phys_addr_t __atags_pointer,
+ unsigned int machine_nr);
#else
-static inline struct machine_desc *
+static inline const struct machine_desc *
setup_machine_tags(phys_addr_t __atags_pointer, unsigned int machine_nr)
{
early_print("no ATAGS support: can't continue\n");
tag->hdr.tag = ATAG_NONE;
}
-struct machine_desc * __init setup_machine_tags(phys_addr_t __atags_pointer,
- unsigned int machine_nr)
+const struct machine_desc * __init
+setup_machine_tags(phys_addr_t __atags_pointer, unsigned int machine_nr)
{
struct tag *tags = (struct tag *)&default_tags;
- struct machine_desc *mdesc = NULL, *p;
+ const struct machine_desc *mdesc = NULL, *p;
char *from = default_command_line;
default_tags.mem.start = PHYS_OFFSET;
* If a dtb was passed to the kernel in r2, then use it to choose the
* correct machine_desc and to setup the system.
*/
-struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
+const struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
{
struct boot_param_header *devtree;
- struct machine_desc *mdesc, *mdesc_best = NULL;
+ const struct machine_desc *mdesc, *mdesc_best = NULL;
unsigned int score, mdesc_score = ~1;
unsigned long dt_root;
const char *model;
DT_MACHINE_START(GENERIC_DT, "Generic DT based system")
MACHINE_END
- mdesc_best = (struct machine_desc *)&__mach_desc_GENERIC_DT;
+ mdesc_best = &__mach_desc_GENERIC_DT;
#endif
if (!dt_phys)
ldrcc pc, [tbl, scno, lsl #2] @ call sys_* routine
add r1, sp, #S_OFF
-2: mov why, #0 @ no longer a real syscall
cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE)
eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back
- bcs arm_syscall
+ bcs arm_syscall
+2: mov why, #0 @ no longer a real syscall
b sys_ni_syscall @ not private func
#if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI)
maps[ARM_SEC_CORE].unw_sec = s;
else if (strcmp(".ARM.exidx.exit.text", secname) == 0)
maps[ARM_SEC_EXIT].unw_sec = s;
+ else if (strcmp(".ARM.exidx.text.unlikely", secname) == 0)
+ maps[ARM_SEC_UNLIKELY].unw_sec = s;
+ else if (strcmp(".ARM.exidx.text.hot", secname) == 0)
+ maps[ARM_SEC_HOT].unw_sec = s;
else if (strcmp(".init.text", secname) == 0)
maps[ARM_SEC_INIT].txt_sec = s;
else if (strcmp(".text", secname) == 0)
maps[ARM_SEC_CORE].txt_sec = s;
else if (strcmp(".exit.text", secname) == 0)
maps[ARM_SEC_EXIT].txt_sec = s;
+ else if (strcmp(".text.unlikely", secname) == 0)
+ maps[ARM_SEC_UNLIKELY].txt_sec = s;
+ else if (strcmp(".text.hot", secname) == 0)
+ maps[ARM_SEC_HOT].txt_sec = s;
}
for (i = 0; i < ARM_SEC_MAX; i++)
continue;
}
- err = request_irq(irq, handler, IRQF_NOBALANCING, "arm-pmu",
+ err = request_irq(irq, handler,
+ IRQF_NOBALANCING | IRQF_NO_THREAD, "arm-pmu",
cpu_pmu);
if (err) {
pr_err("unable to request IRQ%d for ARM PMU counters\n",
__setup("fpe=", fpe_setup);
#endif
-extern void paging_init(struct machine_desc *desc);
+extern void paging_init(const struct machine_desc *desc);
extern void sanity_check_meminfo(void);
extern enum reboot_mode reboot_mode;
-extern void setup_dma_zone(struct machine_desc *desc);
+extern void setup_dma_zone(const struct machine_desc *desc);
unsigned int processor_id;
EXPORT_SYMBOL(processor_id);
static const char *cpu_name;
static const char *machine_name;
static char __initdata cmd_line[COMMAND_LINE_SIZE];
-struct machine_desc *machine_desc __initdata;
+const struct machine_desc *machine_desc __initdata;
static union { char c[4]; unsigned long l; } endian_test __initdata = { { 'l', '?', '?', 'b' } };
#define ENDIANNESS ((char)endian_test.l)
void __init dump_machine_table(void)
{
- struct machine_desc *p;
+ const struct machine_desc *p;
early_print("Available machine support:\n\nID (hex)\tNAME\n");
for_each_machine_desc(p)
}
early_param("mem", early_mem);
-static void __init request_standard_resources(struct machine_desc *mdesc)
+static void __init request_standard_resources(const struct machine_desc *mdesc)
{
struct memblock_region *region;
struct resource *res;
void __init setup_arch(char **cmdline_p)
{
- struct machine_desc *mdesc;
+ const struct machine_desc *mdesc;
setup_processor();
mdesc = setup_machine_fdt(__atags_pointer);
seq_printf(m, "model name\t: %s rev %d (%s)\n",
cpu_name, cpuid & 15, elf_platform);
-#if defined(CONFIG_SMP)
- seq_printf(m, "BogoMIPS\t: %lu.%02lu\n",
- per_cpu(cpu_data, i).loops_per_jiffy / (500000UL/HZ),
- (per_cpu(cpu_data, i).loops_per_jiffy / (5000UL/HZ)) % 100);
-#else
- seq_printf(m, "BogoMIPS\t: %lu.%02lu\n",
- loops_per_jiffy / (500000/HZ),
- (loops_per_jiffy / (5000/HZ)) % 100);
-#endif
/* dump out the processor features */
seq_puts(m, "Features\t: ");
void __init smp_cpus_done(unsigned int max_cpus)
{
- int cpu;
- unsigned long bogosum = 0;
-
- for_each_online_cpu(cpu)
- bogosum += per_cpu(cpu_data, cpu).loops_per_jiffy;
-
- printk(KERN_INFO "SMP: Total of %d processors activated "
- "(%lu.%02lu BogoMIPS).\n",
- num_online_cpus(),
- bogosum / (500000/HZ),
- (bogosum / (5000/HZ)) % 100);
+ printk(KERN_INFO "SMP: Total of %d processors activated.\n",
+ num_online_cpus());
hyp_mode_check();
}
if (tlb_ops_need_broadcast())
on_each_cpu(ipi_flush_tlb_all, NULL, 1);
else
- local_flush_tlb_all();
+ __flush_tlb_all();
broadcast_tlb_a15_erratum();
}
if (tlb_ops_need_broadcast())
on_each_cpu_mask(mm_cpumask(mm), ipi_flush_tlb_mm, mm, 1);
else
- local_flush_tlb_mm(mm);
+ __flush_tlb_mm(mm);
broadcast_tlb_mm_a15_erratum(mm);
}
on_each_cpu_mask(mm_cpumask(vma->vm_mm), ipi_flush_tlb_page,
&ta, 1);
} else
- local_flush_tlb_page(vma, uaddr);
+ __flush_tlb_page(vma, uaddr);
broadcast_tlb_mm_a15_erratum(vma->vm_mm);
}
ta.ta_start = kaddr;
on_each_cpu(ipi_flush_tlb_kernel_page, &ta, 1);
} else
- local_flush_tlb_kernel_page(kaddr);
+ __flush_tlb_kernel_page(kaddr);
broadcast_tlb_a15_erratum();
}
if (tlb_ops_need_broadcast())
on_each_cpu(ipi_flush_bp_all, NULL, 1);
else
- local_flush_bp_all();
+ __flush_bp_all();
}
return regs->ARM_r0;
}
+static long do_cache_op_restart(struct restart_block *);
+
static inline int
-do_cache_op(unsigned long start, unsigned long end, int flags)
+__do_cache_op(unsigned long start, unsigned long end)
{
- struct mm_struct *mm = current->active_mm;
- struct vm_area_struct *vma;
+ int ret;
+ unsigned long chunk = PAGE_SIZE;
+
+ do {
+ if (signal_pending(current)) {
+ struct thread_info *ti = current_thread_info();
+
+ ti->restart_block = (struct restart_block) {
+ .fn = do_cache_op_restart,
+ };
+
+ ti->arm_restart_block = (struct arm_restart_block) {
+ {
+ .cache = {
+ .start = start,
+ .end = end,
+ },
+ },
+ };
+
+ return -ERESTART_RESTARTBLOCK;
+ }
+
+ ret = flush_cache_user_range(start, start + chunk);
+ if (ret)
+ return ret;
+ cond_resched();
+ start += chunk;
+ } while (start < end);
+
+ return 0;
+}
+
+static long do_cache_op_restart(struct restart_block *unused)
+{
+ struct arm_restart_block *restart_block;
+
+ restart_block = ¤t_thread_info()->arm_restart_block;
+ return __do_cache_op(restart_block->cache.start,
+ restart_block->cache.end);
+}
+
+static inline int
+do_cache_op(unsigned long start, unsigned long end, int flags)
+{
if (end < start || flags)
return -EINVAL;
- down_read(&mm->mmap_sem);
- vma = find_vma(mm, start);
- if (vma && vma->vm_start < end) {
- if (start < vma->vm_start)
- start = vma->vm_start;
- if (end > vma->vm_end)
- end = vma->vm_end;
+ if (!access_ok(VERIFY_READ, start, end - start))
+ return -EFAULT;
- up_read(&mm->mmap_sem);
- return flush_cache_user_range(start, end);
- }
- up_read(&mm->mmap_sem);
- return -EINVAL;
+ return __do_cache_op(start, end);
}
/*
--- /dev/null
+/*
+ * Copyright (C) 2013 Uwe Kleine-Koenig for Pengutronix
+ *
+ * This program is free software; you can redistribute it and/or modify it under
+ * the terms of the GNU General Public License version 2 as published by the
+ * Free Software Foundation.
+ */
+#include <linux/io.h>
+#include <linux/reboot.h>
+#include <asm/barrier.h>
+#include <asm/v7m.h>
+
+void armv7m_restart(enum reboot_mode mode, const char *cmd)
+{
+ dsb();
+ __raw_writel(V7M_SCB_AIRCR_VECTKEY | V7M_SCB_AIRCR_SYSRESETREQ,
+ BASEADDR_V7M_SCB + V7M_SCB_AIRCR);
+ dsb();
+}
@ Invalidate the old TLBs
mcr p15, 4, r0, c8, c7, 0 @ TLBIALLH
- dsb
+ dsb ish
eret
mcrr p15, 6, r2, r3, c2 @ Write VTTBR
isb
mcr p15, 0, r0, c8, c3, 0 @ TLBIALLIS (rt ignored)
- dsb
+ dsb ish
isb
mov r2, #0
mov r3, #0
mcr p15, 4, r0, c8, c3, 4
/* Invalidate instruction caches Inner Shareable (ICIALLUIS) */
mcr p15, 0, r0, c7, c1, 0
- dsb
+ dsb ish
isb @ Not necessary if followed by eret
bx lr
for (addr = guest_ipa; addr < end; addr += PAGE_SIZE) {
pte_t pte = pfn_pte(pfn, PAGE_S2_DEVICE);
- kvm_set_s2pte_writable(&pte);
ret = mmu_topup_memory_cache(&cache, 2, 2);
if (ret)
$(obj)/csumpartialcopy.o: $(obj)/csumpartialcopygeneric.S
$(obj)/csumpartialcopyuser.o: $(obj)/csumpartialcopygeneric.S
+
+ifeq ($(CONFIG_KERNEL_MODE_NEON),y)
+ NEON_FLAGS := -mfloat-abi=softfp -mfpu=neon
+ CFLAGS_xor-neon.o += $(NEON_FLAGS)
+ lib-$(CONFIG_XOR_BLOCKS) += xor-neon.o
+endif
--- /dev/null
+/*
+ * linux/arch/arm/lib/xor-neon.c
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/raid/xor.h>
+
+#ifndef __ARM_NEON__
+#error You should compile this file with '-mfloat-abi=softfp -mfpu=neon'
+#endif
+
+/*
+ * Pull in the reference implementations while instructing GCC (through
+ * -ftree-vectorize) to attempt to exploit implicit parallelism and emit
+ * NEON instructions.
+ */
+#if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 6)
+#pragma GCC optimize "tree-vectorize"
+#else
+/*
+ * While older versions of GCC do not generate incorrect code, they fail to
+ * recognize the parallel nature of these functions, and emit plain ARM code,
+ * which is known to be slower than the optimized ARM code in asm-arm/xor.h.
+ */
+#warning This code requires at least version 4.6 of GCC
+#endif
+
+#pragma GCC diagnostic ignored "-Wunused-variable"
+#include <asm-generic/xor.h>
+
+struct xor_block_template const xor_block_neon_inner = {
+ .name = "__inner_neon__",
+ .do_2 = xor_8regs_2,
+ .do_3 = xor_8regs_3,
+ .do_4 = xor_8regs_4,
+ .do_5 = xor_8regs_5,
+};
+++ /dev/null
-/*
- * Debugging macro for DaVinci
- *
- * Author: Kevin Hilman, MontaVista Software, Inc. <source@mvista.com>
- *
- * 2007 (c) MontaVista Software, Inc. This file is licensed under
- * the terms of the GNU General Public License version 2. This program
- * is licensed "as is" without any warranty of any kind, whether express
- * or implied.
- */
-
-/* Modifications
- * Jan 2009 Chaithrika U S Added senduart, busyuart, waituart
- * macros, based on debug-8250.S file
- * but using 32-bit accesses required for
- * some davinci devices.
- */
-
-#include <linux/serial_reg.h>
-
-#include <mach/serial.h>
-
-#define UART_SHIFT 2
-
-#if defined(CONFIG_DEBUG_DAVINCI_DMx_UART0)
-#define UART_BASE DAVINCI_UART0_BASE
-#elif defined(CONFIG_DEBUG_DAVINCI_DA8XX_UART1)
-#define UART_BASE DA8XX_UART1_BASE
-#elif defined(CONFIG_DEBUG_DAVINCI_DA8XX_UART2)
-#define UART_BASE DA8XX_UART2_BASE
-#elif defined(CONFIG_DEBUG_DAVINCI_TNETV107X_UART1)
-#define UART_BASE TNETV107X_UART2_BASE
-#define UART_VIRTBASE TNETV107X_UART2_VIRT
-#else
-#error "Select a specifc port for DEBUG_LL"
-#endif
-
-#ifndef UART_VIRTBASE
-#define UART_VIRTBASE IO_ADDRESS(UART_BASE)
-#endif
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =UART_BASE
- ldr \rv, =UART_VIRTBASE
- .endm
-
- .macro senduart,rd,rx
- str \rd, [\rx, #UART_TX << UART_SHIFT]
- .endm
-
- .macro busyuart,rd,rx
-1002: ldr \rd, [\rx, #UART_LSR << UART_SHIFT]
- and \rd, \rd, #UART_LSR_TEMT | UART_LSR_THRE
- teq \rd, #UART_LSR_TEMT | UART_LSR_THRE
- bne 1002b
- .endm
-
- .macro waituart,rd,rx
-#ifdef FLOW_CONTROL
-1001: ldr \rd, [\rx, #UART_MSR << UART_SHIFT]
- tst \rd, #UART_MSR_CTS
- beq 1001b
-#endif
- .endm
-
+++ /dev/null
-/*
- * arch/arm/mach-dove/include/mach/debug-macro.S
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#include <mach/bridge-regs.h>
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =DOVE_SB_REGS_PHYS_BASE
- ldr \rv, =DOVE_SB_REGS_VIRT_BASE
- orr \rp, \rp, #0x00012000
- orr \rv, \rv, #0x00012000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/* arch/arm/mach-ebsa110/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
-**/
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0xf0000000
- orr \rp, \rp, #0x00000be0
- mov \rp, \rv
- .endm
-
-#define UART_SHIFT 2
-#define FLOW_CONTROL
-#include <asm/hardware/debug-8250.S>
Say 'Y' here if you want your kernel to support the
Vision Engraving Systems EP9307 SoM.
-choice
- prompt "Select a UART for early kernel messages"
-
-config EP93XX_EARLY_UART1
- bool "UART1"
-
-config EP93XX_EARLY_UART2
- bool "UART2"
-
-config EP93XX_EARLY_UART3
- bool "UART3"
-
-endchoice
-
endmenu
endif
+++ /dev/null
-/*
- * arch/arm/mach-ep93xx/include/mach/debug-macro.S
- * Debugging macro include header
- *
- * Copyright (C) 2006 Lennert Buytenhek <buytenh@wantstofly.org>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or (at
- * your option) any later version.
- */
-#include <mach/ep93xx-regs.h>
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =EP93XX_APB_PHYS_BASE @ Physical base
- ldr \rv, =EP93XX_APB_VIRT_BASE @ virtual base
- orr \rp, \rp, #0x000c0000
- orr \rv, \rv, #0x000c0000
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
*((volatile unsigned int *)ptr) = value;
}
-#if defined(CONFIG_EP93XX_EARLY_UART1)
-#define UART_BASE EP93XX_UART1_PHYS_BASE
-#elif defined(CONFIG_EP93XX_EARLY_UART2)
-#define UART_BASE EP93XX_UART2_PHYS_BASE
-#elif defined(CONFIG_EP93XX_EARLY_UART3)
-#define UART_BASE EP93XX_UART3_PHYS_BASE
-#else
-#define UART_BASE EP93XX_UART1_PHYS_BASE
-#endif
-
-#define PHYS_UART_DATA (UART_BASE + 0x00)
-#define PHYS_UART_FLAG (UART_BASE + 0x18)
+#define PHYS_UART_DATA (CONFIG_DEBUG_UART_PHYS + 0x00)
+#define PHYS_UART_FLAG (CONFIG_DEBUG_UART_PHYS + 0x18)
#define UART_FLAG_TXFF 0x20
static inline void putc(int c)
#include <asm/hardware/dec21285.h>
-#ifndef CONFIG_DEBUG_DC21285_PORT
- /* For NetWinder debugging */
- .macro addruart, rp, rv, tmp
- mov \rp, #0x000003f8
- orr \rv, \rp, #0xfe000000 @ virtual
- orr \rv, \rv, #0x00e00000 @ virtual
- orr \rp, \rp, #0x7c000000 @ physical
- .endm
-
-#define UART_SHIFT 0
-#define FLOW_CONTROL
-#include <asm/hardware/debug-8250.S>
-
-#else
#include <mach/hardware.h>
/* For EBSA285 debugging */
.equ dc21285_high, ARMCSR_BASE & 0xff000000
.macro waituart,rd,rx
.endm
-#endif
+++ /dev/null
-/*
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Copyright (C) 2001-2006 Storlink, Corp.
- * Copyright (C) 2008-2009 Paulius Zaleckas <paulius.zaleckas@teltonika.lt>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <mach/hardware.h>
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =GEMINI_UART_BASE @ physical
- ldr \rv, =IO_ADDRESS(GEMINI_UART_BASE) @ virtual
- .endm
-
-#define UART_SHIFT 2
-#define FLOW_CONTROL
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/* arch/arm/mach-integrator/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
-*/
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0x16000000 @ physical base address
- mov \rv, #0xf0000000 @ virtual base
- add \rv, \rv, #0x16000000 >> 4
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/*
- * arch/arm/mach-iop13xx/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0x00002300
- orr \rp, \rp, #0x00000040
- orr \rv, \rp, #0xfe000000 @ virtual
- orr \rv, \rv, #0x00e80000
- orr \rp, \rp, #0xff000000 @ physical
- orr \rp, \rp, #0x00d80000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/mach-iop32x/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0xfe000000 @ physical as well as virtual
- orr \rp, \rp, #0x00800000 @ location of the UART
- mov \rv, \rp
- .endm
-
-#define UART_SHIFT 0
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/mach-iop33x/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0x00ff0000
- orr \rp, \rp, #0x0000f700
- orr \rv, #0xfe000000 @ virtual
- orr \rp, #0xff000000 @ physical
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/* arch/arm/mach-ixp4xx/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
- .macro addruart, rp, rv, tmp
-#ifdef __ARMEB__
- mov \rp, #3 @ Uart regs are at off set of 3 if
- @ byte writes used - Big Endian.
-#else
- mov \rp, #0
-#endif
- orr \rv, \rp, #0xfe000000 @ virtual
- orr \rv, \rv, #0x00f00000
- orr \rp, \rp, #0xc8000000 @ physical
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/mach-kirkwood/include/mach/debug-macro.S
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#include <mach/bridge-regs.h>
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =KIRKWOOD_REGS_PHYS_BASE
- ldr \rv, =KIRKWOOD_REGS_VIRT_BASE
- orr \rp, \rp, #0x00012000
- orr \rv, \rv, #0x00012000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/mach-lpc32xx/include/mach/debug-macro.S
- *
- * Author: Kevin Wells <kevin.wells@nxp.com>
- *
- * Copyright (C) 2010 NXP Semiconductors
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-
-/*
- * Debug output is hardcoded to standard UART 5
-*/
-
- .macro addruart, rp, rv, tmp
- ldreq \rp, =0x40090000
- ldrne \rv, =0xF4090000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/mach-mv78xx0/include/mach/debug-macro.S
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#include <mach/mv78xx0.h>
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =MV78XX0_REGS_PHYS_BASE
- ldr \rv, =MV78XX0_REGS_VIRT_BASE
- orr \rp, \rp, #0x00012000
- orr \rv, \rv, #0x00012000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/mach-orion5x/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
-*/
-
-#include <mach/orion5x.h>
-
- .macro addruart, rp, rv, tmp
- ldr \rp, =ORION5X_REGS_PHYS_BASE
- ldr \rv, =ORION5X_REGS_VIRT_BASE
- orr \rp, \rp, #0x00012000
- orr \rv, \rv, #0x00012000
- .endm
-
-#define UART_SHIFT 2
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/* arch/arm/mach-realview/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifdef CONFIG_DEBUG_REALVIEW_STD_PORT
-#define DEBUG_LL_UART_OFFSET 0x00009000
-#elif defined(CONFIG_DEBUG_REALVIEW_PB1176_PORT)
-#define DEBUG_LL_UART_OFFSET 0x0010c000
-#endif
-
-#ifndef DEBUG_LL_UART_OFFSET
-#error "Unknown RealView platform"
-#endif
-
- .macro addruart, rp, rv, tmp
- mov \rp, #DEBUG_LL_UART_OFFSET
- orr \rv, \rp, #0xfb000000 @ virtual base
- orr \rp, \rp, #0x10000000 @ physical base
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
+++ /dev/null
-/* arch/arm/mach-rpc/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
-*/
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0x00010000
- orr \rp, \rp, #0x00000fe0
- orr \rv, \rp, #0xe0000000 @ virtual
- orr \rp, \rp, #0x03000000 @ physical
- .endm
-
-#define UART_SHIFT 2
-#define FLOW_CONTROL
-#include <asm/hardware/debug-8250.S>
+++ /dev/null
-/*
- * arch/arm/plat-spear/include/plat/debug-macro.S
- *
- * Debugging macro include header for spear platform
- *
- * Copyright (C) 2009 ST Microelectronics
- * Viresh Kumar <viresh.linux@gmail.com>
- *
- * This file is licensed under the terms of the GNU General Public
- * License version 2. This program is licensed "as is" without any
- * warranty of any kind, whether express or implied.
- */
-
-#include <linux/amba/serial.h>
-#include <mach/spear.h>
-
- .macro addruart, rp, rv, tmp
- mov \rp, #SPEAR_DBG_UART_BASE @ Physical base
- mov \rv, #VA_SPEAR_DBG_UART_BASE @ Virtual base
- .endm
-
- .macro senduart, rd, rx
- strb \rd, [\rx, #UART01x_DR] @ ASC_TX_BUFFER
- .endm
-
- .macro waituart, rd, rx
-1001: ldr \rd, [\rx, #UART01x_FR] @ FLAG REGISTER
- tst \rd, #UART01x_FR_TXFF @ TX_FULL
- bne 1001b
- .endm
-
- .macro busyuart, rd, rx
-1002: ldr \rd, [\rx, #UART01x_FR] @ FLAG REGISTER
- tst \rd, #UART011_FR_TXFE @ TX_EMPTY
- beq 1002b
- .endm
/* Debug uart for linux, will be used for debug and uncompress messages */
#define SPEAR_DBG_UART_BASE SPEAR_ICM1_UART_BASE
-#define VA_SPEAR_DBG_UART_BASE VA_SPEAR_ICM1_UART_BASE
/* Sysctl base for spear platform */
#define SPEAR_SYS_CTRL_BASE SPEAR_ICM3_SYS_CTRL_BASE
/* Debug uart for linux, will be used for debug and uncompress messages */
#define SPEAR_DBG_UART_BASE UART_BASE
-#define VA_SPEAR_DBG_UART_BASE VA_UART_BASE
#endif /* SPEAR13XX */
+++ /dev/null
-/* arch/arm/mach-versatile/include/mach/debug-macro.S
- *
- * Debugging macro include header
- *
- * Copyright (C) 1994-1999 Russell King
- * Moved from linux/arch/arm/kernel/debug.S by Ben Dooks
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
-*/
-
- .macro addruart, rp, rv, tmp
- mov \rp, #0x001F0000
- orr \rp, \rp, #0x00001000
- orr \rv, \rp, #0xf1000000 @ virtual base
- orr \rp, \rp, #0x10000000 @ physical base
- .endm
-
-#include <asm/hardware/debug-pl01x.S>
raw_spin_lock_irqsave(&l2x0_lock, flags);
__l2x0_flush_all();
writel_relaxed(0, l2x0_base + L2X0_CTRL);
- dsb();
+ dsb(st);
raw_spin_unlock_irqrestore(&l2x0_lock, flags);
}
outer_cache.disable = l2x0_disable;
}
- printk(KERN_INFO "%s cache controller enabled\n", type);
- printk(KERN_INFO "l2x0: %d ways, CACHE_ID 0x%08x, AUX_CTRL 0x%08x, Cache size: %d B\n",
- ways, cache_id, aux, l2x0_size);
+ pr_info("%s cache controller enabled\n", type);
+ pr_info("l2x0: %d ways, CACHE_ID 0x%08x, AUX_CTRL 0x%08x, Cache size: %d kB\n",
+ ways, cache_id, aux, l2x0_size >> 10);
}
#ifdef CONFIG_OF
.data = (void *)&aurora_no_outer_data},
{ .compatible = "marvell,aurora-outer-cache",
.data = (void *)&aurora_with_outer_data},
- { .compatible = "bcm,bcm11351-a2-pl310-cache",
+ { .compatible = "brcm,bcm11351-a2-pl310-cache",
+ .data = (void *)&bcm_l2x0_data},
+ { .compatible = "bcm,bcm11351-a2-pl310-cache", /* deprecated name */
.data = (void *)&bcm_l2x0_data},
{}
};
add r12, r12, r2
cmp r12, r1
blo 1b
- dsb
+ dsb ishst
icache_line_size r2, r3
sub r3, r2, #1
bic r12, r0, r3
mov r0, #0
ALT_SMP(mcr p15, 0, r0, c7, c1, 6) @ invalidate BTB Inner Shareable
ALT_UP(mcr p15, 0, r0, c7, c5, 6) @ invalidate BTB
- dsb
+ dsb ishst
isb
mov pc, lr
}
/* Queue a TLB invalidate and flush the I-cache if necessary. */
- if (!tlb_ops_need_broadcast())
- cpumask_set_cpu(cpu, &tlb_flush_pending);
- else
- cpumask_setall(&tlb_flush_pending);
+ cpumask_setall(&tlb_flush_pending);
if (icache_is_vivt_asid_tagged())
__flush_icache_all();
if (cpumask_test_and_clear_cpu(cpu, &tlb_flush_pending)) {
local_flush_bp_all();
local_flush_tlb_all();
- if (erratum_a15_798181())
- dummy_flush_tlb_a15_erratum();
}
atomic64_set(&per_cpu(active_asids, cpu), asid);
unsigned end = start + size;
apply_to_page_range(&init_mm, start, size, __dma_update_pte, &prot);
- dsb();
flush_tlb_kernel_range(start, end);
}
* of type casting from pmd_t * to pte_t *.
*/
-pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
-{
- pgd_t *pgd;
- pud_t *pud;
- pmd_t *pmd = NULL;
-
- pgd = pgd_offset(mm, addr);
- if (pgd_present(*pgd)) {
- pud = pud_offset(pgd, addr);
- if (pud_present(*pud))
- pmd = pmd_offset(pud, addr);
- }
-
- return (pte_t *)pmd;
-}
-
struct page *follow_huge_addr(struct mm_struct *mm, unsigned long address,
int write)
{
return 0;
}
-pte_t *huge_pte_alloc(struct mm_struct *mm,
- unsigned long addr, unsigned long sz)
-{
- pgd_t *pgd;
- pud_t *pud;
- pte_t *pte = NULL;
-
- pgd = pgd_offset(mm, addr);
- pud = pud_alloc(mm, pgd, addr);
- if (pud)
- pte = (pte_t *)pmd_alloc(mm, pud, addr);
-
- return pte;
-}
-
-struct page *
-follow_huge_pmd(struct mm_struct *mm, unsigned long address,
- pmd_t *pmd, int write)
-{
- struct page *page;
-
- page = pte_page(*(pte_t *)pmd);
- if (page)
- page += ((address & ~PMD_MASK) >> PAGE_SHIFT);
- return page;
-}
-
int pmd_huge(pmd_t pmd)
{
return pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT);
}
#endif
-void __init setup_dma_zone(struct machine_desc *mdesc)
+void __init setup_dma_zone(const struct machine_desc *mdesc)
{
#ifdef CONFIG_ZONE_DMA
if (mdesc->dma_zone_size) {
return phys;
}
-void __init arm_memblock_init(struct meminfo *mi, struct machine_desc *mdesc)
+void __init arm_memblock_init(struct meminfo *mi,
+ const struct machine_desc *mdesc)
{
int i;
* called function. This means you can't use any function or debugging
* method which may touch any device, otherwise the kernel _will_ crash.
*/
-static void __init devicemaps_init(struct machine_desc *mdesc)
+static void __init devicemaps_init(const struct machine_desc *mdesc)
{
struct map_desc map;
unsigned long addr;
* paging_init() sets up the page tables, initialises the zone memory
* maps, and sets up the zero page, bad page and bad page tables.
*/
-void __init paging_init(struct machine_desc *mdesc)
+void __init paging_init(const struct machine_desc *mdesc)
{
void *zero_page;
* paging_init() sets up the page tables, initialises the zone memory
* maps, and sets up the zero page, bad page and bad page tables.
*/
-void __init paging_init(struct machine_desc *mdesc)
+void __init paging_init(const struct machine_desc *mdesc)
{
early_trap_init((void *)CONFIG_VECTORS_BASE);
mpu_setup();
#endif
mov pc, lr
+/* Suspend/resume support: taken from arch/arm/mm/proc-arm926.S */
+.globl cpu_feroceon_suspend_size
+.equ cpu_feroceon_suspend_size, 4 * 3
+#ifdef CONFIG_ARM_CPU_SUSPEND
+ENTRY(cpu_feroceon_do_suspend)
+ stmfd sp!, {r4 - r6, lr}
+ mrc p15, 0, r4, c13, c0, 0 @ PID
+ mrc p15, 0, r5, c3, c0, 0 @ Domain ID
+ mrc p15, 0, r6, c1, c0, 0 @ Control register
+ stmia r0, {r4 - r6}
+ ldmfd sp!, {r4 - r6, pc}
+ENDPROC(cpu_feroceon_do_suspend)
+
+ENTRY(cpu_feroceon_do_resume)
+ mov ip, #0
+ mcr p15, 0, ip, c8, c7, 0 @ invalidate I+D TLBs
+ mcr p15, 0, ip, c7, c7, 0 @ invalidate I+D caches
+ ldmia r0, {r4 - r6}
+ mcr p15, 0, r4, c13, c0, 0 @ PID
+ mcr p15, 0, r5, c3, c0, 0 @ Domain ID
+ mcr p15, 0, r1, c2, c0, 0 @ TTB address
+ mov r0, r6 @ control register
+ b cpu_resume_mmu
+ENDPROC(cpu_feroceon_do_resume)
+#endif
+
.type __feroceon_setup, #function
__feroceon_setup:
mov r0, #0
add r0, r0, r2
subs r1, r1, r2
bhi 2b
- dsb
+ dsb ishst
mov pc, lr
ENDPROC(cpu_v7_dcache_clean_area)
1:
#endif
-3: mov r10, #0
+ /* Cortex-A15 Errata */
+3: ldr r10, =0x00000c0f @ Cortex-A15 primary part number
+ teq r0, r10
+ bne 4f
+
+#ifdef CONFIG_ARM_ERRATA_773022
+ cmp r6, #0x4 @ only present up to r0p4
+ mrcle p15, 0, r10, c1, c0, 1 @ read aux control register
+ orrle r10, r10, #1 << 1 @ disable loop buffer
+ mcrle p15, 0, r10, c1, c0, 1 @ write aux control register
+#endif
+
+4: mov r10, #0
mcr p15, 0, r10, c7, c5, 0 @ I+BTB cache invalidate
dsb
#ifdef CONFIG_MMU
ENTRY(v7wbi_flush_user_tlb_range)
vma_vm_mm r3, r2 @ get vma->vm_mm
mmid r3, r3 @ get vm_mm->context.id
- dsb
+ dsb ish
mov r0, r0, lsr #PAGE_SHIFT @ align address
mov r1, r1, lsr #PAGE_SHIFT
asid r3, r3 @ mask ASID
add r0, r0, #PAGE_SZ
cmp r0, r1
blo 1b
- dsb
+ dsb ish
mov pc, lr
ENDPROC(v7wbi_flush_user_tlb_range)
* - end - end address (exclusive, may not be aligned)
*/
ENTRY(v7wbi_flush_kern_tlb_range)
- dsb
+ dsb ish
mov r0, r0, lsr #PAGE_SHIFT @ align address
mov r1, r1, lsr #PAGE_SHIFT
mov r0, r0, lsl #PAGE_SHIFT
add r0, r0, #PAGE_SZ
cmp r0, r1
blo 1b
- dsb
+ dsb ish
isb
mov pc, lr
ENDPROC(v7wbi_flush_kern_tlb_range)
ENTRY(vfp_support_entry)
DBGSTR3 "instr %08x pc %08x state %p", r0, r2, r10
+ ldr r3, [sp, #S_PSR] @ Neither lazy restore nor FP exceptions
+ and r3, r3, #MODE_MASK @ are supported in kernel mode
+ teq r3, #USR_MODE
+ bne vfp_kmode_exception @ Returns through lr
+
VFPFMRX r1, FPEXC @ Is the VFP enabled?
DBGSTR1 "fpexc %08x", r1
tst r1, #FPEXC_EN
#include <linux/init.h>
#include <linux/uaccess.h>
#include <linux/user.h>
+#include <linux/export.h>
#include <asm/cp15.h>
#include <asm/cputype.h>
return NOTIFY_OK;
}
+void vfp_kmode_exception(void)
+{
+ /*
+ * If we reach this point, a floating point exception has been raised
+ * while running in kernel mode. If the NEON/VFP unit was enabled at the
+ * time, it means a VFP instruction has been issued that requires
+ * software assistance to complete, something which is not currently
+ * supported in kernel mode.
+ * If the NEON/VFP unit was disabled, and the location pointed to below
+ * is properly preceded by a call to kernel_neon_begin(), something has
+ * caused the task to be scheduled out and back in again. In this case,
+ * rebuilding and running with CONFIG_DEBUG_ATOMIC_SLEEP enabled should
+ * be helpful in localizing the problem.
+ */
+ if (fmrx(FPEXC) & FPEXC_EN)
+ pr_crit("BUG: unsupported FP instruction in kernel mode\n");
+ else
+ pr_crit("BUG: FP instruction issued in kernel mode with FP unit disabled\n");
+}
+
+#ifdef CONFIG_KERNEL_MODE_NEON
+
+/*
+ * Kernel-side NEON support functions
+ */
+void kernel_neon_begin(void)
+{
+ struct thread_info *thread = current_thread_info();
+ unsigned int cpu;
+ u32 fpexc;
+
+ /*
+ * Kernel mode NEON is only allowed outside of interrupt context
+ * with preemption disabled. This will make sure that the kernel
+ * mode NEON register contents never need to be preserved.
+ */
+ BUG_ON(in_interrupt());
+ cpu = get_cpu();
+
+ fpexc = fmrx(FPEXC) | FPEXC_EN;
+ fmxr(FPEXC, fpexc);
+
+ /*
+ * Save the userland NEON/VFP state. Under UP,
+ * the owner could be a task other than 'current'
+ */
+ if (vfp_state_in_hw(cpu, thread))
+ vfp_save_state(&thread->vfpstate, fpexc);
+#ifndef CONFIG_SMP
+ else if (vfp_current_hw_state[cpu] != NULL)
+ vfp_save_state(vfp_current_hw_state[cpu], fpexc);
+#endif
+ vfp_current_hw_state[cpu] = NULL;
+}
+EXPORT_SYMBOL(kernel_neon_begin);
+
+void kernel_neon_end(void)
+{
+ /* Disable the NEON/VFP unit. */
+ fmxr(FPEXC, fmrx(FPEXC) & ~FPEXC_EN);
+ put_cpu();
+}
+EXPORT_SYMBOL(kernel_neon_end);
+
+#endif /* CONFIG_KERNEL_MODE_NEON */
+
/*
* VFP support code initialisation.
*/
return 0;
}
-late_initcall(vfp_init);
+core_initcall(vfp_init);
extern const struct raid6_recov_calls raid6_recov_ssse3;
extern const struct raid6_recov_calls raid6_recov_avx2;
+extern const struct raid6_calls raid6_neonx1;
+extern const struct raid6_calls raid6_neonx2;
+extern const struct raid6_calls raid6_neonx4;
+extern const struct raid6_calls raid6_neonx8;
+
/* Algorithm list */
extern const struct raid6_calls * const raid6_algos[];
extern const struct raid6_recov_calls *const raid6_recov_algos[];
altivec*.c
int*.c
tables.c
+neon?.c
raid6_pq-$(CONFIG_X86) += recov_ssse3.o recov_avx2.o mmx.o sse1.o sse2.o avx2.o
raid6_pq-$(CONFIG_ALTIVEC) += altivec1.o altivec2.o altivec4.o altivec8.o
+raid6_pq-$(CONFIG_KERNEL_MODE_NEON) += neon.o neon1.o neon2.o neon4.o neon8.o
hostprogs-y += mktables
altivec_flags := -maltivec -mabi=altivec
endif
+# The GCC option -ffreestanding is required in order to compile code containing
+# ARM/NEON intrinsics in a non C99-compliant environment (such as the kernel)
+ifeq ($(CONFIG_KERNEL_MODE_NEON),y)
+NEON_FLAGS := -ffreestanding
+ifeq ($(ARCH),arm)
+NEON_FLAGS += -mfloat-abi=softfp -mfpu=neon
+endif
+ifeq ($(ARCH),arm64)
+CFLAGS_REMOVE_neon1.o += -mgeneral-regs-only
+CFLAGS_REMOVE_neon2.o += -mgeneral-regs-only
+CFLAGS_REMOVE_neon4.o += -mgeneral-regs-only
+CFLAGS_REMOVE_neon8.o += -mgeneral-regs-only
+endif
+endif
+
targets += int1.c
$(obj)/int1.c: UNROLL := 1
$(obj)/int1.c: $(src)/int.uc $(src)/unroll.awk FORCE
$(obj)/altivec8.c: $(src)/altivec.uc $(src)/unroll.awk FORCE
$(call if_changed,unroll)
+CFLAGS_neon1.o += $(NEON_FLAGS)
+targets += neon1.c
+$(obj)/neon1.c: UNROLL := 1
+$(obj)/neon1.c: $(src)/neon.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+CFLAGS_neon2.o += $(NEON_FLAGS)
+targets += neon2.c
+$(obj)/neon2.c: UNROLL := 2
+$(obj)/neon2.c: $(src)/neon.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+CFLAGS_neon4.o += $(NEON_FLAGS)
+targets += neon4.c
+$(obj)/neon4.c: UNROLL := 4
+$(obj)/neon4.c: $(src)/neon.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+CFLAGS_neon8.o += $(NEON_FLAGS)
+targets += neon8.c
+$(obj)/neon8.c: UNROLL := 8
+$(obj)/neon8.c: $(src)/neon.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
quiet_cmd_mktable = TABLE $@
cmd_mktable = $(obj)/mktables > $@ || ( rm -f $@ && exit 1 )
&raid6_intx2,
&raid6_intx4,
&raid6_intx8,
+#ifdef CONFIG_KERNEL_MODE_NEON
+ &raid6_neonx1,
+ &raid6_neonx2,
+ &raid6_neonx4,
+ &raid6_neonx8,
+#endif
NULL
};
--- /dev/null
+/*
+ * linux/lib/raid6/neon.c - RAID6 syndrome calculation using ARM NEON intrinsics
+ *
+ * Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/raid/pq.h>
+
+#ifdef __KERNEL__
+#include <asm/neon.h>
+#else
+#define kernel_neon_begin()
+#define kernel_neon_end()
+#define cpu_has_neon() (1)
+#endif
+
+/*
+ * There are 2 reasons these wrappers are kept in a separate compilation unit
+ * from the actual implementations in neonN.c (generated from neon.uc by
+ * unroll.awk):
+ * - the actual implementations use NEON intrinsics, and the GCC support header
+ * (arm_neon.h) is not fully compatible (type wise) with the kernel;
+ * - the neonN.c files are compiled with -mfpu=neon and optimization enabled,
+ * and we have to make sure that we never use *any* NEON/VFP instructions
+ * outside a kernel_neon_begin()/kernel_neon_end() pair.
+ */
+
+#define RAID6_NEON_WRAPPER(_n) \
+ static void raid6_neon ## _n ## _gen_syndrome(int disks, \
+ size_t bytes, void **ptrs) \
+ { \
+ void raid6_neon ## _n ## _gen_syndrome_real(int, \
+ unsigned long, void**); \
+ kernel_neon_begin(); \
+ raid6_neon ## _n ## _gen_syndrome_real(disks, \
+ (unsigned long)bytes, ptrs); \
+ kernel_neon_end(); \
+ } \
+ struct raid6_calls const raid6_neonx ## _n = { \
+ raid6_neon ## _n ## _gen_syndrome, \
+ raid6_have_neon, \
+ "neonx" #_n, \
+ 0 \
+ }
+
+static int raid6_have_neon(void)
+{
+ return cpu_has_neon();
+}
+
+RAID6_NEON_WRAPPER(1);
+RAID6_NEON_WRAPPER(2);
+RAID6_NEON_WRAPPER(4);
+RAID6_NEON_WRAPPER(8);
--- /dev/null
+/* -----------------------------------------------------------------------
+ *
+ * neon.uc - RAID-6 syndrome calculation using ARM NEON instructions
+ *
+ * Copyright (C) 2012 Rob Herring
+ *
+ * Based on altivec.uc:
+ * Copyright 2002-2004 H. Peter Anvin - All Rights Reserved
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, Inc., 53 Temple Place Ste 330,
+ * Boston MA 02111-1307, USA; either version 2 of the License, or
+ * (at your option) any later version; incorporated herein by reference.
+ *
+ * ----------------------------------------------------------------------- */
+
+/*
+ * neon$#.c
+ *
+ * $#-way unrolled NEON intrinsics math RAID-6 instruction set
+ *
+ * This file is postprocessed using unroll.awk
+ */
+
+#include <arm_neon.h>
+
+typedef uint8x16_t unative_t;
+
+#define NBYTES(x) ((unative_t){x,x,x,x, x,x,x,x, x,x,x,x, x,x,x,x})
+#define NSIZE sizeof(unative_t)
+
+/*
+ * The SHLBYTE() operation shifts each byte left by 1, *not*
+ * rolling over into the next byte
+ */
+static inline unative_t SHLBYTE(unative_t v)
+{
+ return vshlq_n_u8(v, 1);
+}
+
+/*
+ * The MASK() operation returns 0xFF in any byte for which the high
+ * bit is 1, 0x00 for any byte for which the high bit is 0.
+ */
+static inline unative_t MASK(unative_t v)
+{
+ const uint8x16_t temp = NBYTES(0);
+ return (unative_t)vcltq_s8((int8x16_t)v, (int8x16_t)temp);
+}
+
+void raid6_neon$#_gen_syndrome_real(int disks, unsigned long bytes, void **ptrs)
+{
+ uint8_t **dptr = (uint8_t **)ptrs;
+ uint8_t *p, *q;
+ int d, z, z0;
+
+ register unative_t wd$$, wq$$, wp$$, w1$$, w2$$;
+ const unative_t x1d = NBYTES(0x1d);
+
+ z0 = disks - 3; /* Highest data disk */
+ p = dptr[z0+1]; /* XOR parity */
+ q = dptr[z0+2]; /* RS syndrome */
+
+ for ( d = 0 ; d < bytes ; d += NSIZE*$# ) {
+ wq$$ = wp$$ = vld1q_u8(&dptr[z0][d+$$*NSIZE]);
+ for ( z = z0-1 ; z >= 0 ; z-- ) {
+ wd$$ = vld1q_u8(&dptr[z][d+$$*NSIZE]);
+ wp$$ = veorq_u8(wp$$, wd$$);
+ w2$$ = MASK(wq$$);
+ w1$$ = SHLBYTE(wq$$);
+
+ w2$$ = vandq_u8(w2$$, x1d);
+ w1$$ = veorq_u8(w1$$, w2$$);
+ wq$$ = veorq_u8(w1$$, wd$$);
+ }
+ vst1q_u8(&p[d+NSIZE*$$], wp$$);
+ vst1q_u8(&q[d+NSIZE*$$], wq$$);
+ }
+}
IS_X86 = yes
endif
+ifeq ($(ARCH),arm)
+ CFLAGS += -I../../../arch/arm/include -mfpu=neon
+ HAS_NEON = yes
+endif
+ifeq ($(ARCH),arm64)
+ CFLAGS += -I../../../arch/arm64/include
+ HAS_NEON = yes
+endif
+
ifeq ($(IS_X86),yes)
OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o
CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \
gcc -c -x assembler - >&/dev/null && \
rm ./-.o && echo -DCONFIG_AS_AVX2=1)
+else ifeq ($(HAS_NEON),yes)
+ OBJS += neon.o neon1.o neon2.o neon4.o neon8.o
+ CFLAGS += -DCONFIG_KERNEL_MODE_NEON=1
else
HAS_ALTIVEC := $(shell echo -e '\#include <altivec.h>\nvector int a;' |\
gcc -c -x c - >&/dev/null && \
raid6test: test.c raid6.a
$(CC) $(CFLAGS) -o raid6test $^
+neon1.c: neon.uc ../unroll.awk
+ $(AWK) ../unroll.awk -vN=1 < neon.uc > $@
+
+neon2.c: neon.uc ../unroll.awk
+ $(AWK) ../unroll.awk -vN=2 < neon.uc > $@
+
+neon4.c: neon.uc ../unroll.awk
+ $(AWK) ../unroll.awk -vN=4 < neon.uc > $@
+
+neon8.c: neon.uc ../unroll.awk
+ $(AWK) ../unroll.awk -vN=8 < neon.uc > $@
+
altivec1.c: altivec.uc ../unroll.awk
$(AWK) ../unroll.awk -vN=1 < altivec.uc > $@
./mktables > tables.c
clean:
- rm -f *.o *.a mktables mktables.c *.uc int*.c altivec*.c tables.c raid6test
+ rm -f *.o *.a mktables mktables.c *.uc int*.c altivec*.c neon*.c tables.c raid6test
spotless: clean
rm -f *~
projects / cascardo / linux.git / commitdiff