Sage Journals: Discover world-class research

Abstract

Many automobile drivers have come to believe that installation of video recording equipment in a car is essential to providing evidence in case of a car accident and to facilitate reporting to police. Thus, it is important to develop an in-vehicle video recorder with high resolution to document accidents. To provide such video quality, the H.264 video format, which offers high compression and good quality, has been used for an in-vehicle video recorder. However, due to problems with fabricated evidence of digital records, we propose a new watermark method to authenticate in-vehicle video. This watermarking scheme authenticates and protects in-vehicle video. The proposed scheme manipulates the quantization table of the I-frames in car surveillance videos and embeds watermarks in the I-frames. Experimental verification was done to confirm the authentication and tamper detection of the proposed watermark image method for in-vehicle video.

Keywords

Smart car black box security video watermark

Introduction

The use of video surveillance recording systems for vehicles has increased considerably in recent years as drivers recognize the potential for providing video evidence to insurance companies and police officers in case of a car accident. In addition, several insurance companies provide a discount of more than 4%–5% for cars equipped with video surveillance recording devices.

Figure 1 shows the system for an in-vehicle black box that records events on a secure digital (SD) card without human intervention to prevent insurance fraud. Sometimes, however, offenders fabricate evidence of an automobile accident to shift evidence in their favor.¹ Counterfeit video scenes from a car accident can lead to erroneous conclusions. Therefore, to address this issue, we investigated the authentication² and tamper detection³ methods for videos produced by an in-vehicle video recorder.

Figure 1.

In-vehicle black box.

Tamper detection² is a branch of the watermarking field and can be categorized as either active authentication or passive authentication (Figure 2).

Figure 2.

Categorization of image authentication.

Previous research has not focused on watermarking methods^2–12 for a car video recorder. Digital watermarking can be used to protect video frames in a car video. The key benefits of watermarking over other techniques are imperceptibility and invisibility.

Requirements for a watermark include the quality and robustness of watermarked video frames, and the quality should be near that of original video frames. Meanwhile, robustness refers to the ability to extract a watermark after alteration of watermarked video frames. Despite such advantages, there are currently no watermark algorithms based on video frames from an in-vehicle video recorder. Therefore, the design of watermark techniques for a vehicle video recorder is an opportunity for development. Over many years, different types of watermarking algorithms for image and video protection have appeared.^4–12

A new watermarking technique was suggested⁵ based on compressed videos; however, the scheme did not provide security for strong watermarking with watermarking locality. A watermarking scheme⁶ that provided robustness and maintained video quality at different video bit rates was proposed. A watermarking algorithm was also proposed^7,8 that maintained good quality, but could not cope with an increase in the bit rate. In order to decrease the bit rate, suitable macro-blocks were selected, which were a set of non-zero coefficients. Other schemes concealed the watermark in discrete cosine transform (DCT) quantized coefficients.^10,11 Bartolini et al.¹² suggested a scheme that concealed watermarks in AC coefficients, but required many DCT computations. A watermarking¹³ scheme for video verification was designed based on the hybrid feature of least significant bits (LSB).¹⁴ Transform domain methods changed the coefficients of the DCT^10,11 and discrete wavelet transform (DWT)¹⁵ transform domains and distributed the watermark signal energy to all pixels.

A watermarking method was suggested that concealed a watermark in DCT coefficients and motion vectors.¹⁶ This watermarking method provided a greater payload without significantly degrading video quality. An innovative semi-fragile watermarking method was suggested¹⁷ that concealed a watermark in DCT AC values in H.264 video. An unusual watermarking method was suggested for content-based video certification.¹⁸ A method to find tampered parts using a fragile watermark based on a hybrid feature has also been introduced.¹⁹ A watermark scheme has been put forth that maintains the secrecy of video content²⁰ and allows efficiency for secure transmission. A lossless watermarking method²¹ was proposed with little overhead and good visual quality based on intra-prediction.

Our study focused on the application of a watermarking method to videos from a video recorder in a vehicle. Such a method allows determination of whether video evidence is a forgery, and thus assists in determining the facts of a vehicle accident and facilitates insurance processing.

In this article, a watermark was concealed in a 4 × 4 block composed of luminance pixels from the DCT block of a video frame. Based on experimentation, a tampered region of a watermarked video frame was identified that had been changed and it was possible to extract a watermark from watermarked video frames. In addition, the authenticity of the original video (including watermark) stored in a vehicle camera was verified.

Section “Proposed system” of this article presents the proposed scheme. Section “Experimental results” provides the results of experimental procedures, and Section “Conclusion” concludes with a discussion of the limitations and benefits of the proposed method.

Proposed system

Figure 3 shows the general schematic diagram of the proposed scheme. An H.264 bit stream is transferred to a video server via a wireless network²² in a car or other vehicle. Next, inverse entropy is used for extraction of video information from the H.264 bit stream. In the next step, an I-slice is extracted through parsing of the video stream and then Y (luminance) is extracted from the I-slice. In a concealing procedure, each Y block of a frame is divided into numerous blocks that will be converted into a DCT domain. Then, the watermark is concealed in a set of frames based on adjusting associations of DCT coefficients after choosing appropriate blocks.

Figure 3.

General watermarking scheme for H.264 based on DCT coefficients.

Entropy coding is then performed. The result is a novel watermarked video bit stream. All essential data for verification are concealed in the H.264 and cannot be lost.

Design concept

Looking at JPEG coefficients, numerous zeros can be seen in frequency components after DCT quantization. A coefficient A_xy of a DCT is defined in Figure 4 where x and y are indices in DCT {0 ≤ (x and y) < 4} and coefficients are used for a domain to conceal hidden bits.

Figure 4.

Regular sequence in coefficients of quantized DCT.

$ϕ$ is an array variable for watermarking or hidden bits, where ϕ has values of ϕ(1) = A₀₀, ϕ(2) = A₀₁, ϕ(3) = A₁₀, …, ϕ(xy) = A₃₃. A watermark is composed of a sequence of bit streams that can be hidden in $ϕ_{1}^{xy}$ by manipulation of values in ϕ. A watermarking rate p (0 ≤ p ≤ 1) is defined in equation (1), where l_ϕ is the length of ϕ, and a number of n pixels can be successfully embedded into these continuous zeros in $ϕ$ . The capacity and quality of our proposed scheme will determine the value of p

n = ⌈ l_{\emptyset} \times p ⌉

(1)

Proposed scheme

The cover frame of a car video is fragmented into non-overlapping blocks using a decomposition method with a block size of 4 × 4 pixels. Each block of frames is transformed into DCT coefficients using a standard quantization table, and then a watermark is concealed in the coefficient as a regular sequence.

Embedding and extracting the watermark of the proposed scheme are shown in sections “Embedding procedure” and “Extracting procedure,” respectively.

Embedding procedure

The proposed embedding procedure for a car video frame is depicted in Figure 5. A binary watermark composed of “0” or “1” is embedded in a number of blocks of a frame. The mid-frequency coefficients of this DCT are altered according to the value of a binary watermark in the embedding procedure. After the embedding procedure, reconstruction of the block from the quantized DCT block using “inverse DCT” is required.

Figure 5.

Block diagram for a watermark embedding algorithm.

Input. Video frame I, watermark W, m: index of each pixel of the watermark, watermarking ratio p, block indices: x, y.

Output. Watermarked frame I′.

Step 1. Divide a video frame I into (H × W)/(4 × 4) non-overlapping blocks in a block B_j, where 1 ≤ j ≤ (H × W)/(4 × 4). The initial variable m is {1 ≤ m ≤ 4 × 4}.

Step 2. Obtain a block A_j, by FDCT(B_j), which is a forward DCT transform. In order to prepare for the embedding watermark, the value of $A_{j}$ is assigned to ϕ, that is, $ϕ_{m} = A_{j} (x, y)$ .

Step 3. Find the end coefficient (i.e. ϕ > 0) in A where i = 1, …, 4 × 4. If $ϕ_{i}$ is not equal to 0, k = i. If there is no zero coefficient in ϕ, k = 0. Calculate n = (16 − k) × p. If k = 0, then stop and go to step 2, otherwise, $ϕ_{i} = ϕ_{i} + 1$ and move to the next step.

Step 4. Repeat n: (equation (2)) where w is a pixel in the watermark. Then, move ϕ to A as follows: $A_{j} (x, y) = ϕ_{k}$

\emptyset_{k}^{n} = {\begin{matrix} 0, & if w_{m} = 0 \\ 1 or - 1, & if w_{m} = 1 \end{matrix}

(2)

Step 5. Obtain blocks B_j, by IDCT(A_j), which is an inverse DCT transform.

Step 6. Integrate all blocks B_j to obtain a watermarked frame I′. If 1 ≤ j ≤ (H × W)/16, go to step 2, otherwise move to the next step.

Step 7. Complete and return I′.

Extraction procedure

The proposed extraction procedure for a car video frame is shown in Figure 6 where mid-frequency coefficients are selected and used for reconstruction of the watermark. After the extraction procedure, reconstruction of a block of the frame using “inverse DCT” is required.

Figure 6.

Block diagram for a watermark extraction algorithm.

Input. Watermarked frame I′, m: index of a watermark, watermarking ratio p, block indices: x, y.

Output. Watermark W′.

Step 1. Divide a video frame I′ into (H × W)/(4 × 4) non-overlapping blocks in a block B_j, where (x and y) equal to 4 and 1 ≤ j ≤ (H × W)/(4 × 4). The initial variable m is {1 ≤ m ≤ 4 × 4}.

Step 2. Obtain block A_j, by FDCT(B_j), which is a forward DCT transform. In order to prepare for extraction of a watermark, the value of $A_{j}$ is assigned to ϕ, that is, $ϕ_{m} = A_{j} (x, y)$ .

Step 3. Find the end coefficient (i.e. ϕ > 1 or ϕ < 1) in A. If $ϕ_{i} > 1$ or $ϕ_{i} < 1$ , then k = i, where i = 4 × 4, …, 1. Calculate n = (16 − k) × p. If k = 0, break and go to step 2, Otherwise move to the next step.

Step 4. Repeat n: (equation (3)). In order to restore original block A, equation (4) is executed with block A. Repeat n: (equation (4)). Then, move ϕ to A as follows: $A_{j} (x, y) = ϕ_{k}$

W_{m} = {\begin{matrix} 0 & if ϕ_{k} = 0 \\ 1 & if ϕ_{k} = 1 or - 1 \end{matrix}

(3)

\emptyset_{k}^{n} = {\begin{matrix} nothing & if ϕ (k) = 0 \\ 0 & if ϕ (k) = 1 or - 1 \end{matrix}

(4)

Step 5. Obtain blocks B_j, by IDCT(A_j), which is an inverse DCT transform. If 1 ≤ j ≤ (H × W)/16, go to step 2, otherwise move to the next step.

Step 6. Integrate all the blocks B_j to obtain a cover image I. If 1 ≤ j ≤ (H × W)/16, go to step 2, otherwise move to the next step.

Step 7. Return watermark W′.

Step 8. If W ≠ W′ and normalized coefficient (NC) < 0.9 (equation (7)), the watermarked video frame has tampered pixels.

Evaluation method

Experiments were conducted to conceal a watermark, to evaluate the quality of video frames, to detect forgery, and to test the performance of different attacks based on wireless surveillance devices in a car or other vehicle in order to demonstrate the proposed scheme. In-vehicle video sequences used in experiments with the proposed scheme were 1280 × 720 in size with 80 frames. A binary watermarking image size of 110 × 70 was used. Quality evaluation of a frame was based on the peak signal-to-noise ratio (PSNR)¹⁴ value of the YUV channel of a frame

PSNR = 10 \times \log_{10} (\frac{255^{2}}{MSE})

(5)

where mean squared error (MSE) is found as

MSE = \frac{1}{MN} \sum_{i = 1}^{M} \sum_{j = 1}^{N} [b (i, j) - b' (i, j)]^{2}

(6)

where b and b′ are pixels of the original and handled frames, respectively, and M × N is the size of the frames. The PSNR is used to evaluate an image and video using simple arithmetic operation such as equation (5) instead of a human visual system. If PSNR values of frames are higher than 35 dB, the watermarked videos are acceptable values, but they would be damaged. Reconstructed watermarks could be used for authentication of these videos.

Experimental results

The results from experiments using the proposed scheme for achieving video authentication found that a watermark was concealed in a car video for tamper detection. The watermark extracted from video frames indicated whether video frames were forged. Table 1 lists the configuration parameters of the experiments.

Table 1.

Configuration bounds for encoding.

Profile	Baseline
Frames	80
Frame rate	30 fps
Number of reference frames	10
RD optimization	Disabled
Rate control	Disabled

RD: rate distortion.

In this article, original video frames and a watermark image were required. Figures 7 and 8 show the video frames, and Figure 9 shows the watermark used to protect privacy to and authenticate the original video frame.

Figure 7.

Original car video frame: road#1.

Figure 8.

Original car video frame: road#2.

Figure 9.

Watermark image (110 × 70).

Figure 10 shows the watermarked video frame embedding a watermark image from Figure 9. As can be seen from Figures 7 and 10, discrimination between the original frame in Figure 7 and the watermarked video frame is difficult for human visual perception.

Figure 10.

Watermarked video frame.

To estimate the quality of video frames, PSNR values of 80 frames were used. Figure 11 shows the PSNR results, which were designed to maintain a value of more than 50 dB.

Figure 11.

PSNR values of the scheme with frames.

The PSNR value of watermarked video frames was dependent on the embedding rate (or capacity). Thus, the balance between the quality of a video frame and the embedding rate can be adjusted for different purposes. The proposed scheme returned a PSNR value approximately 0.12 dB better than the value from a previous study.¹⁹

The watermark in an image or video frame has a negative impact on an original image or video frame. Thus, it is important to determine the influence of watermarking on video frames. The NC value in equation (7) indicates the resemblance between an original watermark and an extracted watermark

NC = \frac{\sum_{i = 0}^{M} \sum_{j = 0}^{N} w_{i, j} {\hat{w}}_{i, j}}{\sum_{i = 0}^{M} \sum_{j = 0}^{N} {[w_{i, j}]}^{2}}

(7)

In equation (7), $w_{i, j}$ and ${\hat{w}}_{i, j}$ are an original and an extracted watermark, respectively, where M and N denote the width and height of the watermark. When NC = “1,” the original watermark is the same as the extracted watermark. Thus, in case of NC ≥ 0.7, acceptability is achieved in aspect of a quality, but the video watermarked frame may be modified by somebody.

Figure 12 indicates the practical relationship between the two factors of NC value and bit rate. The NC value was little changed with a precipitous increase in bit rate. The result of Figure 12 shows that compression with frames does not impact NC values; thus, the proposed scheme is robust against compression with frames.

Figure 12.

Influence of compression with frames with bit rate increases from 500 to 3000.

Different attack strategies, including pepper and salt noise, on video frames were investigated in order to measure the proposed scheme strength. For noise attack analysis, random noise was generated and distributed over frames as indicated in Figure 13(a). Figure 13(b) shows the video frames that include random noise. Figure 13(c) is the extracted watermark image from the watermarked video frame. The NC value of Figure 13(c) was 0.9975. Thus, the watermark derived directly from Figure 13(c) was not significantly altered by attack.

Figure 13.

Noise attack on a video frame: (a) watermarked frame, (b) noisy frame derived from a watermarked frame, and (c) extracted watermark, NC = 0.9775.

A copy and paste attack experiment was applied to a watermarked video frame in Figure 14(a) under the assumption that the car license plate in the watermarked video frame shown in Figure 14(a) was fabricated with license plate “4127” by an attacker. Tamper detection is shown in Figure 14(b) where the car license plate is painted black, which indicates a video frame forgery in Figure 14(a).

Figure 14.

Tamper detection based on a watermarked video frame: (a) tampered video frame and (b) tamper detection.

A performance comparison of the proposed scheme and previous watermarking schemes is shown in Table 2. The criteria of the performance in Table 2 are payload, quality, robustness, etc. “Format compliant” denotes video frame compatibility with standard codecs. Table 2 shows that the proposed scheme was strong in aspects of payload, quality, robustness, and computational complexity, compared to the other two schemes evaluated.

Table 2.

Performance comparison with previous schemes.

Properties	Schemes
Properties	20	21	Proposed scheme
Payload	L	A	A
Quality	L	L	H
Robustness	A	A	H
Computational complexity	L	L	A
Security	H	H	H
Format compliance	Y	Y	Y

L: low; H: high; A: average; Y: yes.

The proposed scheme can be used to solve the problem of forgery in-vehicle video using a watermark authentication technique.

Conclusion

A watermarking scheme to protect in-vehicle video was presented. Tamper detection was robust. Therefore, the proposed scheme is suitable for use in the black box of a smart car. The limitation of our proposed scheme is not high embedding rate. Future work will explore enhancement of the visual quality and payload of the proposed scheme.

Footnotes

Academic Editor: Kye-Shin Lee

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2015R1D1A1A01059253). This work was supported under the framework of the international cooperation program managed by the National Research Foundation of Korea (2016K2A9A2A05005255, FY20**).

References

Park

DW.

Forensic analysis technique of car black box. Int J Softw Eng Appl 2014; 8(11): 1–10.

Wang

. Self-authentication watermark by digital halftoning. In: Proceedings of the 2015 international Carnahan conference on security technology (ICCST), Taipei, Taiwan, 21–24 September 2015, pp.1–6. New York: IEEE.

Hong

. Adaptive fragile watermark for image authentication with tampering localization. In: Proceedings of the 2nd international conference on anti-counterfeiting, security and identification, Guiyang, China, 20–23 August 2008, pp.22–25. New York: IEEE.

Kim

Yang

CN.

Watermark with DSA signature using predictive coding. Multimed Tools Appl 2015; 74(14): 5189–5203.

Noorkami

Mersereau

. Compressed domain video watermarking scheme for H.264. IEEE Image Proc 2005; 2: 890–893.

Golikeri

Nasiopoulos

Wang

. An improved scalar quantization-based digital video watermarking scheme for H.264/AVC. In: Proceedings of the IEEE international symposium on circuits and systems, Island of Kos, Greece, 21–24 May 2006, pp.81–84. New York: IEEE.

Zhang

ATS

. Robust digital image-in-video watermarking for the emerging H.264/AVC standard. In: Proceedings of the IEEE workshop on signal processing systems design and implementation, 2–4 November 2005, pp.657–662. New York: IEEE.

Jiang

Niu

. An innovative video watermarking scheme based on H.264 for copyright protection. Int J Digit Content Technol Appl 2012; 6(5): 290–297.

Bergkvist

. Implementation of a watermarking algorithm for H.264 video sequence, 2004, pp.1–21, http://www.diva-portal.org/smash/get/diva2:19683/FULLTEXT01.pdf

10.

Zhang

ATS

Qiu

. Robust video watermarking of H.264/AVC. IEEE T Circuits Syst 2007; 54(2): 205–209.

11.

Sakazawa

Takishima

Nakajima

H.264 native video watermarking method. In: Proceedings of the IEEE international symposium on circuits and systems, Island of Kos, Greece, 21–24 May 2006, pp.143–149. New York: IEEE.

12.

Bartolini

Manetti

Piva

. A data hiding approach for correcting errors in H.263 video transmitted over a noisy channel. In: Proceedings of the IEEE workshop on multimedia signal processing, Cannes, 3–5 October 2001, pp.65–70. New York: IEEE.

13.

Jiang

Wang

. A solution of video semi-fragile watermarking of authentication based on binary characteristic strings. In: Proceedings of the 2009 international conference on multimedia information networking and security, vol. 2, Wuhan, China, 18–20 November 2009, pp.167–170. New York: IEEE.

14.

Kim

Data hiding by an improved exploiting modification direction. Multimed Tools Appl 2014; 69(3): 569–584.

15.

Alenizi

Kurdahi

Eltawil

. DWT-based watermarking technique for video authentication. In: Proceedings of the 2015 IEEE international conference on electronics, circuits, and systems (ICECS), Cairo, Egypt, 6–9 December 2015, pp.41–44. New York: IEEE.

16.

Qiu

Marziliano

ATS

. A hybrid watermarking scheme for H.264/AVC video. In: Proceedings of the 17th international conference on pattern recognition (ICPR 2004), Cambridge, UK, 23–26 August 2004, pp.865–868. New York: IEEE.

17.

Chen

Lin

. H.264 video authentication based on semi-fragile watermarking. In: Proceedings of the 4th international conference on intelligent information hiding and multimedia signal processing, Harbin, China, 15–17 August 2008, pp.659–662. New York: IEEE.

18.

Wang

. A novel watermarking scheme for H.264/AVC video authentication. Signal Process Image Comm 2011; 26(6): 267–279.

19.

Liu

Zhu

Huang

. A hybrid watermarking scheme for video authentication. In: Proceedings of the 2006 international conference on image processing, Atlanta, Georgia, USA, 8–11 October 2006, pp.2569–2572. New York: IEEE.

20.

Lian

Liu

Ren

. Commutative encryption and watermarking in video compression. IEEE T Circ Syst Vid 2007; 17(6): 774–778.

21.

Park

Shin

. Combined scheme of encryption and watermarking in H.264/Scalable Video Coding (SVC). In: Tsihrintzis

Virvou

Howlett

. (eds) New directions in intelligent interactive multimedia. Berlin: Springer-Verlag Berlin Heidelberg, 2008, pp.351–361.

22.

Tuohy

Glavin

Hughes

. Intra-vehicle networks: a review. IEEE T Intell Transp 2015; 16(2): 534–545.

Secure protection of video recorder video in smart car

Abstract

Keywords

Introduction

Proposed system

Design concept

Proposed scheme

Embedding procedure

Extraction procedure

Evaluation method

Experimental results

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References