前言

2022年的互联网到处弥漫着裁员的消息，自研人脸做为公司在节流侧的重点项目；自研人脸主要涉及移动端SDK，后端解密视频流数据以及算法侧三块；本文重点分析移动端SDK的开发。

1、使用camera2自定义相机（API 21）

camera2的使用可以参考Google的demo，编码优美: https://github.com/googlearchive/android-Camera2Video

圆形头像可以参考，https://github.com/wangshengyang1996/GLCameraDemo

1.1、圆形的相机实现

SurfaceView和GLSurfaceView，TextureView的差异自己搜索下，大部分直接选TextureView就可以了。

圆形的TextureView上菜:

    public class RoundTextureView extends TextureView {
    private static final String TAG = "CustomTextureView";
    private int radius = 0;

    public RoundTextureView(Context context, AttributeSet attrs) {
        super(context, attrs);
        setOutlineProvider(new ViewOutlineProvider() {
            @Override
            public void getOutline(View view, Outline outline) {
                Rect rect = new Rect(0, 0, view.getMeasuredWidth(), view.getMeasuredHeight());
                outline.setRoundRect(rect, radius);
            }
        });
        setClipToOutline(true);
    }

    public void turnRound() {
        invalidateOutline();
    }

    public void setRadius(int radius) {
        this.radius = radius;
    }

    public int getRadius() {
        return radius;
    }
}

是不是惊掉下巴:
1、setClipToOutline(true)设置为true
2、再设置一个ViewOutlineProvider的impl就搞定；

经验: 其实这个方法可以设置任意的圆形控件，像发现宝藏一样，太久没写UI了哈哈；

1.2、打开相机实现预览

这里要简单说下权限问题，其实录制视频虽然需要保存文件到手机其实不用访问SD卡的权限；只需要把保存文件的目录设置在 /storage/emulated/0/Android/data/com.example.android.camera2video/files

上菜

 
context?.getExternalFilesDir(null)

onResume中打开camera

override fun onResume(){
    ...
        
    if (textureView.isAvailable) {
            Log.i(TAG, "111 textureView.isAvailable is ${textureView.isAvailable}")
            openCamera(textureView.width, textureView.height)
        } else {
            Log.i(TAG, "222 textureView.isAvailable is ${textureView.isAvailable}")
            textureView.surfaceTextureListener = surfaceTextureListener
    }
    ...
}

openCamera的实现

这里有个控制线程安全的类Semaphore，译做信号量；可以控制获取资源的线程数量，更加灵活。

Semaphore CountDownLatch CyclicBarrier分别适用不同的多线程场景，简单说明:

Semaphore 可以同时让多个线程同时访问共享资源，通过 acquire() 获取一个许可，如果没有就等待，而 release() 释放一个许可。

CountDownLatch适用多线程下执行依赖顺序的任务，比如有一个任务A，它要等待其他4个任务执行完毕之后才能执行。

CyclicBarrier让一组线程等待至某个状态之后再全部同时执行。

 private val cameraOpenCloseLock = Semaphore(1)
 
 private fun openCamera(width: Int, height: Int) {

        if (!hasPermissionsGranted(VIDEO_PERMISSIONS)) {
            requestVideoPermissions()
            return
        }

        val cameraActivity = this
        if (cameraActivity.isFinishing) return

        val manager = cameraActivity.getSystemService(Context.CAMERA_SERVICE) as CameraManager
        try {
            if (!cameraOpenCloseLock.tryAcquire(2500, TimeUnit.MILLISECONDS)) {
//                throw RuntimeException("Time out waiting to lock camera opening.")
            }

            if (manager.cameraIdList.isEmpty()) {
                throw RuntimeException("Cannot get available cameraList")
            }

            var cameraId = manager.cameraIdList[0]

            for (id in manager.cameraIdList) {
                val characteristics = manager.getCameraCharacteristics(id)

                val direction = characteristics.get(CameraCharacteristics.LENS_FACING)
                if (direction != null &&
                    direction == CameraCharacteristics.LENS_FACING_FRONT
                ) {
                    cameraId = id
                    break
                }
            }

            // Choose the sizes for camera preview and video recording
            val characteristics = manager.getCameraCharacteristics(cameraId)

            val map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                ?: throw RuntimeException("Cannot get available preview/video sizes")
            sensorOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION) ?: 0
            videoSize = chooseVideoSize(map.getOutputSizes(MediaRecorder::class.java))
            previewSize = chooseOptimalSize(
                map.getOutputSizes(SurfaceTexture::class.java),
                width, height, videoSize
            )

            configureTransform(width, height)
            mediaRecorder = MediaRecorder()
            manager.openCamera(cameraId, stateCallback, null)
        } catch (e: CameraAccessException) {
            showToast("Cannot access the camera.")
            cameraActivity.finish()
        } catch (e: NullPointerException) {
            // Currently an NPE is thrown when the Camera2API is used but not supported on the
            // device this code runs.
        } catch (e: InterruptedException) {
            Logger.i("Interrupted while trying to lock camera opening.")
        }
    }
    
     private fun startPreview() {
        if (mCameraDevice == null || !textureView.isAvailable) return

        try {
            closePreviewSession()
            val texture = textureView.surfaceTexture
            previewRequestBuilder =
                mCameraDevice!!.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)

            val previewSurface = Surface(texture)
            previewRequestBuilder.addTarget(previewSurface)

            mCameraDevice?.createCaptureSession(
                listOf(previewSurface),
                object : CameraCaptureSession.StateCallback() {

                    override fun onConfigured(session: CameraCaptureSession) {
                        captureSession = session
                        updatePreview()
                    }

                    override fun onConfigureFailed(session: CameraCaptureSession) {
                        showToast("Failed")
                    }
                }, backgroundHandler
            )
        } catch (e: CameraAccessException) {
            Logger.i(e.toString())
        }

    }
    
     private val stateCallback = object : CameraDevice.StateCallback() {

        override fun onOpened(cameraDevice: CameraDevice) {
            cameraOpenCloseLock.release()
            mCameraDevice = cameraDevice
            startPreview()
            configureTransform(textureView.width, textureView.height)
        }

        override fun onDisconnected(cameraDevice: CameraDevice) {
            cameraOpenCloseLock.release()
            cameraDevice.close()
            mCameraDevice = null
        }

        override fun onError(cameraDevice: CameraDevice, error: Int) {
            cameraOpenCloseLock.release()
            cameraDevice.close()
            mCameraDevice = null
            finish()
        }
    }

1.3、视频录制

准备好MediaRecorder；

protected fun setUpMediaRecorder() {

        if (nextVideoAbsolutePath.isNullOrEmpty()) {
            nextVideoAbsolutePath = getVideoFilePath(this)
        }

        val rotation = windowManager.defaultDisplay.rotation
        when (sensorOrientation) {
            SENSOR_ORIENTATION_DEFAULT_DEGREES ->
                mediaRecorder?.setOrientationHint(DEFAULT_ORIENTATIONS.get(rotation))
            SENSOR_ORIENTATION_INVERSE_DEGREES ->
                mediaRecorder?.setOrientationHint(INVERSE_ORIENTATIONS.get(rotation))
        }

        mediaRecorder?.apply {
            setVideoSource(MediaRecorder.VideoSource.SURFACE)
            setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
            setOutputFile(nextVideoAbsolutePath)
            // 8000表示低采样，48000表示高采样，单位是HZ
//            setVideoEncodingBitRate(8000)
            // 电视是每秒30帧，12-15帧足以表示运动
            setVideoFrameRate(13)
            setVideoSize(videoSize.width, videoSize.height)
            setVideoEncoder(MediaRecorder.VideoEncoder.H264)
            prepare()
        }
    }

开始录制视频；

private fun startRecordingVideo() {
        if (mCameraDevice == null || !textureView.isAvailable) return

        try {
            closePreviewSession()
            setUpMediaRecorder()
            val texture = textureView.surfaceTexture

            val previewSurface = Surface(texture)
            val recorderSurface = mediaRecorder!!.surface
            val surfaces = ArrayList<Surface>().apply {
                add(previewSurface)
                add(recorderSurface)
            }
            previewRequestBuilder =
                mCameraDevice!!.createCaptureRequest(CameraDevice.TEMPLATE_RECORD).apply {
                    addTarget(previewSurface)
                    addTarget(recorderSurface)
                }

            mCameraDevice?.createCaptureSession(
                surfaces,
                object : CameraCaptureSession.StateCallback() {

                    override fun onConfigured(cameraCaptureSession: CameraCaptureSession) {
                        captureSession = cameraCaptureSession
                        updatePreview()
                        runOnUiThread {
                            isRecordingVideo = true
                            mediaRecorder?.start()
                        }
                    }

                    override fun onConfigureFailed(cameraCaptureSession: CameraCaptureSession) {
                        showToast("Failed")
                    }
                }, backgroundHandler
            )
        } catch (e: CameraAccessException) {
            Log.e(TAG, e.toString())
        } catch (e: IOException) {
            Log.e(TAG, e.toString())
        }

    }

停止录制视频；

    
    private fun stopRecordingVideo() {
        try {
            isRecordingVideo = false
            mediaRecorder?.apply {
                stop()
                reset()
            }
        } catch (e: Throwable) {
            Log.i("xxxx,", "stopRecordingVideo error is ${e.message}")
        }

    }
    

2、视频数据加密

主要分五个步骤: 1.生成aes 128位秘钥 2.使用aes将视频数组加密，盐为生成的秘钥 3.将aes秘钥通过下发的rsa公钥加密并base64编码 4.将加密后的视频数组base64编码，上传加密后的aes密钥以及视频编码

2.1、生成aes 128位密钥

网上生成密钥的方法很多，这里用生成固定长度的字符串做实现

    /**
     * 生成16字节的AES密钥（128位就是16字节)
     */
    fun genAESSecret(length:Int = 16): String {
        val random = Random()
        val stringBuffer = StringBuffer()
        for (i in 0 until length) {
            val number: Int = random.nextInt(3)
            var result: Int = 0
            when (number) {
                0 -> {
                    result = (Math.random() * 25 + 65).roundToInt()
                    stringBuffer.append(result.toChar())
                }
                1 -> {
                    result = (Math.random() * 25 + 97).roundToInt()
                    stringBuffer.append(result.toChar())
                }
                2 -> stringBuffer.append(Random().nextInt(10).toString())
            }
        }
        return stringBuffer.toString()
    }

2.2、使用aes将视频数组加密，盐为生成的秘钥

   // 读取文件字节流
   fun fileToByteArray(file: File): ByteArray? {
        try {
            if (!file.exists()) return null

            val fis  = FileInputStream(file)
            val baos = ByteArrayOutputStream()
            val buffer = ByteArray(1024)
            var count = 0
            while (fis.read(buffer).also { count = it } >= 0) {
                baos.write(buffer, 0, count) //读取输入流并写入输出字节流中
            }
            fis.close() //关闭文件输入流
            return baos.toByteArray()
        } catch (e: Throwable) {
            Log.i("xxxx", "fileBase64String() error is " + e.message)
        }
        return null
    }
    
    /**
     * AES加密算法
     */
    fun encryptAES(byteContent: ByteArray, password: String): String {

        var encryptResult: ByteArray? = null
        try {
            val keyByte = password.toByteArray()
            // 密钥
            val secretKey = SecretKeySpec(keyByte, "AES")
            // 算法/模式/填充
            val cipher = Cipher.getInstance("AES/CBC/PKCS5Padding")
            // 加盐，相同内容加密后相同
            val zeroIv = IvParameterSpec(keyByte)
            cipher.init(Cipher.ENCRYPT_MODE, secretKey,zeroIv)
            encryptResult = cipher.doFinal(byteContent)

            return Base64.encodeToString(encryptResult, Base64.DEFAULT)
        } catch (e: Exception) {
            Log.i("xxxx,", "encrypt error ${e.message}")
            e.printStackTrace()
        }

        return ""
    }

2.3、将aes秘钥通过下发的rsa公钥加密并base64编码

这里需要注意的是，RSA加密数据有着严格的长度限制( javax.crypto.IllegalBlockSizeException: input must be under 256 bytes)

    /**
     * RSA对加密的明文有长度的限制，2048位的密钥加密的明文不能超过256字节
     */
    fun encryptRSA(data: String, publicKey: String?): String {
        try {
            if (publicKey.isNullOrEmpty()) {
                return data
            }
            val cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding")
            cipher.init(Cipher.ENCRYPT_MODE, loadPublicKey(publicKey))
            return Base64.encodeToString(cipher.doFinal(data.toByteArray()),Base64.DEFAULT)
        } catch (e: Throwable) {
            Log.i("xxxx", "encryptRSA error is ${e::class.java.name}")
            Log.i("xxxx", "encryptRSA error is ${e.message}")
        }
        return data
    }

2.5、iOS在RSA加密跟Java的一致性解读

iOS在开发过程中一直表示RSA加解密跟Java不兼容，直到我新建一个OC工程和简单的Android工程做对比；发现其实是一致的，原文以字符串”hello world!”为例子

iOS demo地址: https://github.com/ideawu/Objective-C-RSA/blob/master

公钥和私钥对来自iOS的开源项目的demo

const val originString = "hello world!"

const val pubkey = "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDDI2bvVLVYrb4B0raZgFP60VXY\ncvRmk9q56QiTmEm9HXlSPq1zyhyPQHGti5FokYJMzNcKm0bwL1q6ioJuD4EFI56D\na+70XdRz1CjQPQE3yXrXXVvOsmq9LsdxTFWsVBTehdCmrapKZVVx6PKl7myh0cfX\nQmyveT/eqyZK1gYjvQIDAQAB"

const val privkey = "MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAMMjZu9UtVitvgHS\ntpmAU/rRVdhy9GaT2rnpCJOYSb0deVI+rXPKHI9Aca2LkWiRgkzM1wqbRvAvWrqK\ngm4PgQUjnoNr7vRd1HPUKNA9ATfJetddW86yar0ux3FMVaxUFN6F0KatqkplVXHo\n8qXubKHRx9dCbK95P96rJkrWBiO9AgMBAAECgYBO1UKEdYg9pxMX0XSLVtiWf3Na\n2jX6Ksk2Sfp5BhDkIcAdhcy09nXLOZGzNqsrv30QYcCOPGTQK5FPwx0mMYVBRAdo\nOLYp7NzxW/File//169O3ZFpkZ7MF0I2oQcNGTpMCUpaY6xMmxqN22INgi8SHp3w\nVU+2bRMLDXEc/MOmAQJBAP+Sv6JdkrY+7WGuQN5O5PjsB15lOGcr4vcfz4vAQ/uy\nEGYZh6IO2Eu0lW6sw2x6uRg0c6hMiFEJcO89qlH/B10CQQDDdtGrzXWVG457vA27\nkpduDpM6BQWTX6wYV9zRlcYYMFHwAQkE0BTvIYde2il6DKGyzokgI6zQyhgtRJ1x\nL6fhAkB9NvvW4/uWeLw7CHHVuVersZBmqjb5LWJU62v3L2rfbT1lmIqAVr+YT9CK\n2fAhPPtkpYYo5d4/vd1sCY1iAQ4tAkEAm2yPrJzjMn2G/ry57rzRzKGqUChOFrGs\nlm7HF6CQtAs4HC+2jC0peDyg97th37rLmPLB9txnPl50ewpkZuwOAQJBAM/eJnFw\nF5QAcL4CYDbfBKocx82VX/pFXng50T7FODiWbbL4UnxICE0UBFInNNiWJxNEb6jL\n5xd0pcy9O2DOeso="

取两次Java平台用公钥对原文做加密后的结果，可见同一个公钥加密相同的密文结果是不一样的。

         const val enStrJava1 = "DmUfGzD+H1o00rpNuZu9sEbdaT+CCQt7OrrUBFJ0QHNgSawhmJoSsWp5lddkL4jpyfC80S/1nQf0\n" +
                "    6u2mctRrxnOM+gkJtZULrz6Fw0IFz7cgHDuse047AEl/kbartyym6Cf503Gonw0CiJ9yhlrgp8Q0\n" +
                "    gnkZDEM5fBdm4AID4m0="
        const val enStrJava = "USbshGzusjt8IXnhw9pu7yxqeVB5KDvzlKJnwE9WBMvrkTXFqScqk8fd7ATCdmyGUSTbT3bHpdam\n" +
                "    DsTnBbcvxbbqG+QLibYAPTZHt5/1MnHrHeld/n4lB46IRm/z+7oyu8I3S1mJkDcwlLrlA5dTzjlQ\n" +
                "    oHXekIRAPmTj/ZhFfGw="

取两次OC平台用公钥对原文做加密后的结果

        const val enStrObjective_c = "LcbVE7nbzf2Sj/pgLRnSs67e9BWyAp0x3SktRSwob42A1XBFsHQR80/j07BNDrZ/85tuJh6qQD1hB3AgEYacv/+pJEurRz/JUF5e/pslnW/ahT/rudC6U7scjr/7lMbQ1bCDLN9i1oHgQB8qVo3ETEf4jDckzLtw2ZWHS1Gj0pY="
        const val enStrObjective_c1 = "sjQt/VEQruvESSqCKw6/Q9Uv1CR/KGKYFWSxEZwJvGBmyErQPKfYvgfqeeTjUShV9cidVwR4nY9zVzKlkwk2AoAKQxzwGRXWyorKFITSalcQ3wSfZ3hVorBn6QF8eKSH7RYustBDk4I1V5xI+ZPADUQ0srG9J/9acC1l3FL9fKU="
       

上述四条密文用私钥做解密均成功，结果为原文”hello world!”

2.6、Java后端使用RSA私钥解密的实现

  fun decryptRSAPrivate(data: String, privateKey: String?): String? {
        try {
            if (privateKey.isNullOrEmpty()) {
                return data
            }
           val byteData = Base64.decode(data, Base64.DEFAULT)
            val cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding")
            cipher.init(Cipher.DECRYPT_MODE, loadPrivateKey(privateKey))
            return String(cipher.doFinal(byteData))
        } catch (e: Throwable) {
            Log.i("xxxx", "decryptRSA error is ${e.message}")
        }

        return null
    }
    
    
     private fun loadPrivateKey(keyBase64: String?): RSAPrivateKey? {
        val keyBytes: ByteArray = Base64.decode(keyBase64,Base64.DEFAULT)
        val keySpec = PKCS8EncodedKeySpec(keyBytes)
        try {
            val keyFactory = KeyFactory.getInstance("RSA")
            return keyFactory.generatePrivate(keySpec) as RSAPrivateKey
        } catch (e: java.lang.Exception) {
            Log.i("xxxx", "loadPrivateKey error is ${e.message}")
        }
        return null
    }

总结

其实自定义人脸识别本身不是很复杂，端侧的活体检测等其实可以交给算法侧完成；
温习了一遍对称加密和非对称加密，进一步巩固了加减密算法的基础。

自研人脸初探

人脸识别

前言

1、使用camera2自定义相机（API 21）

1.1、圆形的相机实现

1.2、打开相机实现预览

1.3、视频录制

2、视频数据加密

2.1、生成aes 128位密钥

2.2、使用aes将视频数组加密，盐为生成的秘钥

2.3、将aes秘钥通过下发的rsa公钥加密并base64编码

2.5、iOS在RSA加密跟Java的一致性解读

2.6、Java后端使用RSA私钥解密的实现

总结

CATALOG

FEATURED TAGS

FRIENDS

前言

1、使用camera2自定义相机（API 21）

1.1、 圆形的相机实现

1.2、 打开相机实现预览

1.3、视频录制

2、视频数据加密

2.1、生成aes 128位密钥

2.2、使用aes将视频数组加密，盐为生成的秘钥

2.3、将aes秘钥通过下发的rsa公钥加密并base64编码

2.5、iOS在RSA加密跟Java的一致性解读

2.6、Java后端使用RSA私钥解密的实现

总结

CATALOG

FEATURED TAGS

FRIENDS

1.1、圆形的相机实现

1.2、打开相机实现预览