crypto: hisilicon/qm - check function qp num before alg register

When the Kunpeng accelerator executes tasks such as encryption
and decryption have minimum requirements on the number of device
queues. If the number of queues does not meet the requirement,
the process initialization will fail. Therefore, the driver checks
the number of queues on the device before registering the algorithm.
If the number does not meet the requirements, the driver does not register
the algorithm to crypto subsystem, the device is still added to the
qm_list.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - ensure private key less than n

The private key of the curve key size generated by stdrng, which maybe
not less than n. Therefore, the private key with the curve key size
minus 1 is generated to ensure that the private key is less than n.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - Set DMA alignment explicitly

This driver has been implicitly relying on kmalloc alignment
to be sufficient for DMA. This may no longer be the case with
upcoming arm64 changes.

This patch changes it to explicitly request DMA alignment from
the Crypto API.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - Use helper to set reqsize

The value of reqsize must only be changed through the helper.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - support get algs by the capability register

The value of qm algorithm can change dynamically according to the
value of the capability register.

Add xxx_set_qm_algs() function to obtain the algs that the
hardware device supported from the capability register and set
them into usr mode attribute files.

Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Wenkai Lin <linwenkai6@hisilicon.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - optimize registration of ecdh

Use table to store the different ecdh curve configuration,
making the registration of ecdh clearer and expansion more
convenient.

Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - support hpre capability

Read some hpre device configuration info from capability
register, instead of fixed macros.

Signed-off-by: Zhiqi Song <songzhiqi1@huawei.com>
Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - get qp num and depth from hardware registers

Hardware V3 and later versions can obtain qp num and depth supported
by the hardware from registers. To be compatible with later hardware
versions, get qp num and depth from registers instead of fixed marcos.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - don't use GFP_KERNEL to alloc mem during softirq

The hpre encryption driver may be used to encrypt and decrypt packets
during the rx softirq, it is not allowed to use GFP_KERNEL.

Fixes: c8b4b477079d ("crypto: hisilicon - add HiSilicon HPRE accelerator")
Signed-off-by: Zhengchao Shao <shaozhengchao@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - fix memory leak in hpre_curve25519_src_init()

hpre_curve25519_src_init() allocates memory for 'ptr' before calling
memcmp(). If memcmp() returns 0, the function will return '-EINVAL'
without freeing memory.

Signed-off-by: Weili Qian <qianweili@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - use swap() to make code cleaner

Use the macro 'swap()' defined in 'include/linux/minmax.h' to avoid
opencoding it.

Reported-by: Zeal Robot <zealci@zte.com.cn>
Signed-off-by: Yang Guang <yang.guang5@zte.com.cn>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - register ecdh NIST P384

Register ecdh NIST P384 curve and add the tfm initialization.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add check before gx modulo p

The result of gx modulo p is zero if gx is equal to p, so return
error immediately if gx is equal to p.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - fix ecdh self test issue

When the key length is zero, use stdrng to generate private key
to pass the crypto ecdh-nist-p256 self test on vector 2.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add 'default' for switch statement

Return error immediately if it goto 'default' path.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - use 'GENMASK' to generate mask value

Use 'GENMASK' to generate mask value, just make the code clearer.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - delete rudundant initialization

Delete rudundant variable initialization.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - remove the macro of 'HPRE_DEV'

Remove complex macro of 'HPRE_DEV' and replace with the initialized
device pointer.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - fix unmapping invalid dma address

Currently, an invalid dma address may be unmapped when calling
'xx_data_clr_all' in error path, so check dma address of sqe in/out
if initialized before calling 'dma_free_coherent' or 'dma_unmap_single'.

Fixes: a9214b0b6ed2 ("crypto: hisilicon - fix the check on dma address")
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - extend 'cra_driver_name' with curve name

Currently,'cra_driver_name' cannot be used to specify ecdh algorithm
with a special curve, so extending it with curve name.

Fixes: 6763f5ea2d9a ("crypto: ecdh - move curve_id of ECDH from ...")
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add debug log

When the received sqe is abnormal, the error message in the sqe written
back by the hardware is printed to help to analyze the abnormal causes.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - use the correct variable type

The return value of 'le32_to_cpu' is unsigned, so change the
variable type from 'int' to 'unsigned int'.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - delete the rudundant space after return

There are two spaces after return, just keep one.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - fix a typo in hpre_crypto.c

Do a trivial typo fix.
s/discribed/described

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Ruiqi Gong <gongruiqi1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - Add processing of src_data in 'CURVE25519'

HPRE can only deal with src_data smaller than 'p' in 'CURVE25519'
algorithm, but 'rfc7748' says:
'Implementations MUST accept non-canonical values and process them as
if they had been reduced modulo the field prime'
So we get its modulus to p, and then deal it with HPRE.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - fix "hpre_ctx_init" resource leak

When calling "hpre_ctx_set" fails, stop and put qp,
otherwise will leak qp resource.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - fix the check on dma address

System may be able to get physical address of zero if not reserved by
firmware.

The dma address obtained by 'dma_alloc_coherent' is valid, since already
checking cpu va before, so do not check again.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - optimise 'hpre_algs_register' error path

There is redundant code especially when registing new algorithms
in the future.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - delete wrap of 'CONFIG_CRYPTO_DH'

'CRYPTO_DH' has selected in 'Kconfig', so delete 'CONFIG_CRYPTO_DH'.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add 'CURVE25519' algorithm

Enable 'CURVE25519' algorithm in Kunpeng 930.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add 'ECDH' algorithm

1. Enable 'ECDH' algorithm in Kunpeng 930;
2. HPRE ECDH Support: ecdh-nist-p192, ecdh-nist-p256.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add algorithm type

Algorithm type is brought in to get hardware HPRE queue
to support different algorithms.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add version adapt to new algorithms

A new generation of accelerator Kunpeng930 has appeared, and the
corresponding driver needs to be updated to support some new
algorithms of Kunpeng930. To be compatible with Kunpeng920, we
add parameter 'struct hisi_qm *qm' to sec_algs_(un)register to
identify the chip's version.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - fix a bug in dh algorithm

Using 'g' not equal to 2 in dh algorithm may cause an error like this:

arm-smmu-v3 arm-smmu-v3.1.auto: event 0x10 received:
dh: Party A: generate public key test failed. err -22
11375.065672] dh alg: dh: test failed on vector 1, err=-22
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000790000000010
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000120800000080
hpre-dh self test failed
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000000
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000000
arm-smmu-v3 arm-smmu-v3.1.auto: event 0x10 received:
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000790000000010
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000120800000083
arm-smmu-v3 arm-smmu-v3.1.auto: 0x00000000000000c0
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000000
arm-smmu-v3 arm-smmu-v3.1.auto: event 0x10 received:
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000790000000010
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000120800000081
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000040
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000000
arm-smmu-v3 arm-smmu-v3.1.auto: event 0x10 received:
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000790000000010
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000120800000082
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000080
arm-smmu-v3 arm-smmu-v3.1.auto: 0x0000000000000000
hisi_hpre 0000:79:00.0: dat_rd_poison_int_set [error status=0x8] found
hisi_hpre 0000:79:00.0: ooo_rdrsp_err_int_set [error status=0xfc00] found
hisi_hpre 0000:79:00.0: Controller resetting...
hisi_hpre 0000:79:00.0: Controller reset complete
{2}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
{2}[Hardware Error]: event severity: recoverable
{2}[Hardware Error]: Error 0, type: recoverable
{2}[Hardware Error]: section type: unknown, c8b328a8-9917-4af6-9a13-2e08ab2e7586
{2}[Hardware Error]: section length: 0x4c

as we missed initiating 'msg->in'.

Fixes: c8b4b477079d("crypto: hisilicon - add HiSilicon HPRE accelerator")
Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - adjust some coding style

Adjust some coding style to make code aligned.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - remove useless code

1. Remove unused member 'debug_root' in 'struct hpre_debug';
2. The u64 cast is redundant in 'cpu_to_le64'.

Fixes: 848974151618("crypto: hisilicon - Add debugfs for HPRE")
Fixes: dadbe4c11753("crypto: hisilicon/hpre - update debugfs ...")
Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/qm - fix the process of register algorithms to crypto

When the devices are removed or not existing, the corresponding algorithms
which are registered by 'hisi-zip' driver can't be used.

Move 'hisi_zip_register_to_crypto' from 'hisi_zip_init' to
'hisi_zip_probe'. The algorithms will be registered to crypto only when
there is device bind on the driver. And when the devices are removed,
the algorithms will be unregistered.

In the previous process, the function 'xxx_register_to_crypto' need a lock
and a static variable to judge if the registration is the first time.
Move this action into the function 'hisi_qm_alg_register'. Each device
will call 'hisi_qm_alg_register' to add itself to qm list in probe process
and registering algs when the qm list is empty.

Signed-off-by: Yang Shen <shenyang39@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - add debugfs for Hisilicon HPRE

Add debugfs to provides IO operation debug information
and add BD processing timeout count function

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon/hpre - Optimize finding hpre device process

Optimize finding hpre device process according to priority of numa
distance.

Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Shukun Tan <tanshukun1@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add branch prediction macro

This branch prediction macro on the hot path can improve
small performance(about 2%) according to the test.

Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - adjust hpre_crt_para_get

Reorder the input parameters of hpre_crt_para_get to
make it cleaner.

Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Fixed some tiny bugs of HPRE

1.Use memzero_explicit to clear key;
2.Fix some little endian writings;
3.Fix some other bugs and stuff of code style;

Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - Bugfixed tfm leak

1.Fixed the bug of software tfm leakage.
2.Update HW error log message.

Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

crypto: hisilicon - add HiSilicon HPRE accelerator

The HiSilicon HPRE accelerator implements RSA and DH algorithms. It
uses Hisilicon QM as interface to CPU.

This patch provides PCIe driver to the accelerator and registers its
algorithms to crypto akcipher and kpp interfaces.

Signed-off-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>