Effects of feature selection on lane-change maneuver recognition: an analysis of naturalistic driving data Open Access

Lane-change (LC) accidents are accounting for about 4-10 per cent of all crashes (Barr and Najm, 2001) and 1.5 per cent of all motor vehicle fatalities in the USA (N. H. T. S. Administration, 2015). With the development of advanced driver assistance systems, functions such as lane departure warning (LDW) and lane-change assist (LCA) hit the market to help avoid LC related accidents (Visvikis et al., 2008). One of the key problems is how to correctly recognize driver’s LC maneuver in advance. When an improper LC maneuver is occurring, the driver assistance systems should either give warnings or assist this person aborting the maneuver.

Supervised learning is popularly used for recognizing LC maneuvers. This method raise the challenges of how to select the most contributive or efficient features. Longitudinal features [e.g. time to collision, TTC, (Liebner et al., 2013) (Peng et al., 2015), longitudinal acceleration] and lateral features [e.g. steering angle (Xu et al., 2012), yaw rate (Sivaraman and Trivedi, 2014; Doshi et al., 2011), lateral acceleration (Boubezoul et al., 2009; Kasper et al., 2012)] have been used, with the assumption that they are strong enough for LC maneuver recognition by either intuition or empirical knowledge; however, this assumption is still hanging on and yet comprehensively studied.

In general, LC maneuver can be either discretionary or mandatory. A mandatory LC will occur when a driver must leave a lane due to a lane drop or bypass a blockage, etc. A discretionary lane change occurs when a driver prefers a more efficient adjacent lane (J2944, 2013), for example, passing a slow-moving leading vehicle to maintain the current speed (Lee et al., 2004). So it would require different weighted features to recognize LC maneuver with discretionary and mandatory LC cases. Leonhardt and Wanielik evaluated the effects of various features in different LC driving scenarios (Leonhardt and Wanielik, 2017) and showed that even for the same feature, the weight of overtaking a slow vehicle and merging is different. Thus, feature selection process should also take LC scenarios into account.

In this paper, we propose a feature selection method for predicting driver LC behavior. Our aim is to comprehensively figure out different types of features in LC maneuvers and select the most contributive features over various LC scenarios. The main contribution of our work can be summarized as follows:

• presenting a feature selection method from the perspective of statistics to investigate the statistical significance of each feature based on big naturalistic driving data;

• both time-domain and frequency-domain features are considered to fill in gaps in existing works on feature selection; and

• taking different driving scenarios into consideration in the feature extraction procedure to comprehensively evaluate the extracted features.

The remainder of this paper is organized as follow. Section 2 reviews the related work of feature selection and LC maneuver recognition. Section 3 describes how to model the contextual traffic in each LC scenario. Section 4 details data processing and feature extraction. Section 5 shows the results of feature selection and model performance. Section 6 makes conclusion and discussion.

Machine learning techniques, such as support vector machines (SVM) (Mandalia and Salvucci, 2005; Kumar et al., 2013), Naive Bayes (NB), Decision Tree (DT), k-nearest neighbor (KNN) (Lethaus et al., 2013), artificial neural networks (ANN) (Peng et al., 2015) and Bayesian Networks (BN) (Kasper et al., 2012; Li et al., 2016; Weidl et al., 2018), have been implemented to recognize driver LC maneuvers based on a well-trained classifier using labeled datasets. Then new data are fed to the classifier to determine the classification of either LC or LK maneuver. In this way, data being classified as LC means the driver at the moment is prone to make LC maneuver, otherwise not.

Although most papers have made comparison work to show their effectiveness, it is achieved by using the less contributive features in their proposed model. To overcome this bias, we evaluate model performance using the identical features – the comprehensively selected features – and then give them a relatively objective rate to evaluate their contribution for maneuver recognition.

The goal of feature selection is to reduce the dimension of training datasets by removing redundant information. In general, feature selection methods can be grouped into filter and wrapper methods. Filter methods analyze the intrinsic properties of data, ranking and selecting features without involving learning algorithms. On the contrary, the wrapper method with learning algorithms would get involved to score a given subset of features (Guyon et al., 2008). For wrapper methods, the ranking of features can vary from model to model. Here, we shall find the intrinsic properties of feature candidates related to LC and select the most contributive features rather than ranking and selecting features for a specific learning approach. Therefore, the filter method was selected in this paper.

For LC maneuver recognition, the data collected from sensors are time series data, and the properties of the features in the time domain are the most frequently extracted (Liebner et al., 2013) (Kasper et al., 2012). On the other hand, frequency-domain features have already been used to recognize driver state, for instance, the power spectrum features via wavelet transform were selected for Belief networks (Hajinoroozi et al., 2015; Chen et al., 2015). In other areas of time series recognition such as speech recognition (Thomas et al., 2008) and anomaly detection (Zhang et al., 2008), frequency-domain features play an important role.

In this paper, we consider the properties of the features in both time domain and frequency domain to select the most contributive features for LC maneuver recognition.

To capture the most contributive features in each LC scenario, we first model the contextual traffic of the ego vehicle. A potential field diagram (Woo et al., 2016) composed of bubbles with different dynamic sizes was used to describe the dynamic relationship between the ego vehicle and its surrounding vehicles. However, it is not an intuitive way for driving situation analysis. Leonhardt and Wanielik (2017) developed a probabilistic situation assessment model to judge the safety state of the ego vehicle with its surrounding vehicles; however, it only works as behavior recognition model with a single input.

To easily describe the relationship between the ego vehicle and its surroundings, one of the most popular approaches is to segment the surrounding traffic into cell grids. The occupancy state of each cell is represented by a binary value, i.e. occupied or empty (Kasper et al., 2012; Do et al., 2017). We model contextual traffic also based on the cell grid method which is detailed in the next section.

In Do et al. (2017), nine cells and 32 cases (25) were considered in the driving contextual traffic. But the authors did not give the specific boundary of the cells. Kasper and Weidl modeled the cell by carrying the speed-dependent information when a cell will be occupied or will become free (Kasper et al., 2012). But they assumed that the vehicle can move unobstructed toward certain cell, which cannot be satisfied in some situation where a car wants to overtake the ego car. In this paper, we model the cell grid by considering the dynamic relationship between the ego car and the surrounding cars. A three-cell grid is used to model contextual traffic where the ego vehicle executes the LC maneuver, for both left and right LC with eight cases.

As limited by our on-board sensors, which can only detect the traffic in front of the ego vehicle, the traffic situation on back of the ego vehicle is not considered. Despite such limitation, our method of modeling contextual traffic can be extended to more cell grids which can include the traffic on back of the ego vehicle. Here we only model the contextual traffic in front of the ego vehicle as is depicted in Figure 1(a).

We adopt the theory presented in Karim et al. (2013) to define the middle cell (cell_m) and theory in Kesting and Treiber (2013) to define the left (cell_l)/right cell (cell_r). The dynamic length of each cell is $s1*$⁠, $s2*$⁠, $s3*$⁠, as shown in Figure 1(a). The length of cell_m is defined by a mean safe time gap (MSTG) in Karim et al. (2013) as:

where BT_EV and BT_OV are the brake time of the ego vehicle and object vehicle 1, respectively, RT is the driver’s perception-reaction time and for certain vehicle, the BT is calculated by an empirical equation:

where v is the vehicle speed and thus:

where Ṙ is the range rate between the ego vehicle and object vehicle 1. So, the dynamic length of $s1*$ can be written as:

where v is the longitudinal speed of the ego vehicle.

We define cell_l and cell_r based on the Intelligent Driver Model (IDM) (Kesting and Treiber, 2013). Here, the safe distance is derived from the leading vehicle, driving at a desired speed, or preferring accelerations to be within a comfortable range. Additionally, kinematic aspects are taken into account, such as the quadratic relation between braking distance and speed. First, on the left and right lane, desired distances on the left (⁠$sl*$⁠) and right (⁠$sr*$⁠) lane are defined respectively as:

where s₀ is the minimum (bumper-to-bumper) gap, T is the safe time gap, a^* and b^* are acceleration and comfortable deceleration. R_l, Ṙ_l and R_r, Ṙ_r are the range and range rate the ego vehicle with object Vehicle 2 and Vehicle 3 in Figure 1(b), respectively. The dynamic terms $Ṙl⋅Rl/(2⋅a*⋅b*)$ and $Ṙr⋅Rr/(2⋅a*⋅b*)$ imply the intelligent braking strategy for LLC and RLC cases.

Second, based on the desired distance on the left (⁠$sl*$⁠) and right (⁠$sr*$⁠) lane, the dynamic safety distance, namely, the length of $s2*$ and $s3*$⁠, can be written as:

where az is the longitudinal acceleration of the ego vehicle, Δa is the LC threshold and a_bias represents the asymmetric property of LLC and RLC.

All the values of the parameters in equations (1) and (5)-(8) are listed in Table I (Kesting and Treiber, 2013), and the occupancy states of cells can be given as (Figure 2):

Depending on the occupancy state of cell grid, eight scenarios (four scenarios for LLC) can be generated, as depicted in Figure 3:

• LLC Scenario 0_0: When the ego vehicle makes LLC, there are no object vehicles on both cell_m and cell_l;

• LLC Scenario 0_1: When the ego vehicle makes LLC, there is no object vehicle on cell_l but cell_m is occupied;

• LLC Scenario 1_0: When the ego vehicle makes LLC, there is no object vehicle on cell_m but cell_l is occupied;

• LLC Scenario 1_1: When the ego vehicle makes LLC, both cell_m and cell_l are occupied;

• RLC Scenario 0_0: When the ego vehicle makes RLC, there are no object vehicles on both cell_m and cell_r;

• RLC Scenario 0_1: When the ego vehicle makes LLC, there is no object vehicle on cell_m but cell_r is occupied;

• RLC Scenario 1_0: When the ego vehicle makes LLC, there is no object vehicle on cell_r but cell_m is occupied; and

• RLC Scenario 1_1: When the ego vehicle makes LLC, both cell_m and cell_r are occupied.

Here, the name of the LC scenarios such as Scenario 0_1 and Scenario 1_0 is in accordance with the binary states of the occupancy cells illustrated in Figure 3.

The naturalistic driving data that used in this paper are from the project of the Safety Pilot Model Deployment (SPMD).

The on-road test includes multi-modal traffic, hosting approximately 3,000 vehicles equipped with vehicle-to-vehicle (V2V) communication devices (Henclewood et al., 2014). The data sets we used were extracted from 20 vehicles, driving in the field test including 75 miles of roadway, see Figure 3. Roads that marked as yellow are the route SPMD vehicle driving. Drivers voluntarily joined in SPMD project. They drove the SPMD vehicle completely based on their own driving styles with no restriction on their driving behaviors. Each SPMD vehicle was equipped with data acquisition systems (DAS) such as CAN and GPS and vision system such as Mobileye. All the signals coming from different DAS were time-synchronized and were recorded at 10 Hz.

Finally, 1,375 LC cases (761 LLC and 614 RLC) were analyzed. The distribution of the LC cases with respect to the corresponding LC scenarios in Figure 3 can be seen in Table II. We can see that for LLC, most of the cases took place in LLC Scenario 0_0 (365 cases) and LLC Scenario 0_1 (354 cases). For RLC, the dominating cases are RLC Scenario 0_0 (371 cases) and RLC Scenario 1_0 (214 cases). This result implies that when the driver want to execute left/right LC, he/she tends to wait until the destination lane being empty (cell_l/cell_r is unoccupied).

Vehicle yaw rate and lateral acceleration are usually used as strong features of vehicle lateral behavior. Together with longitudinal acceleration, the above signals are necessary for recognition, prediction and modeling vehicle lateral behaviors (Leonhardt and Wanielik, 2017; Higgs and Abbas, 2015; Li et al., 2015; Luo et al., 2016). Here we also choose the following signals directly collected from on-board sensors as our candidate features:

• yaw Rate_t = yaw rate of the ego vehicle at time t;

• az_t = longitudinal acceleration of the ego vehicle at time t; and

• ax_t = lateral acceleration of the ego vehicle at time t.

Time-to-collision (TTC) is the time required for two vehicles to collide if they continue at their present speeds on the same path. It is usually used to evaluate collision risk (Kusano and Gabler, 2011). If a small TTC indicates the driver may execute LC to overtake the slow leading vehicle. Thus the TTC can be regarded as a valuable feature to recognize LC maneuver (Kasper et al., 2012). Time-to-lane crossing (TLC) represents the time available for a driver until the moment at which any part of the vehicle reaches one of the lane boundaries (Godthelp et al., 1984). It is a parameter to estimate if the ego vehicle is going to cross the lane. Based on (J2944, 2013), TTC and TLC are given by.

• TTC with the object vehicle in front on the current lane (TTC_t) at time t:

  $TTCt=RṘ$

  (12)

where R and Ṙ [in Figure 1(b)] are the range and the range rate between the front edge of the ego vehicle and rear edge of the closest object vehicle in the same traveling path as the ego vehicle, respectively. Here, what needs to be mentioned is that TTC is only calculated for the LC case when Cell_m = 1, because Cell_m = 0 means there is no vehicle in the cell.

• TLC at time t (TLC_t):

  $TLCt=dxvx$

  (13)

where dx is lateral distance between the front wheel and the lane boundary of the ego vehicle and vx is the lateral speed.

In case that Ṙ and vx are equal to zero, equations (12) and (13) approach infinity, we use the inverse of $TTCt−1$ and $TLCt−1$ instead.

Vehicle on-board signals are time series, using time-window (TW) for feature extraction is effective to capture the information during the past few seconds (Thissen et al., 2003; Salfner and Malek, 2007). In the case of LC recognition, different length of TW between 1 and 5 s are selected for feature extraction (Mandalia and Salvucci, 2005). To capture the properties of time series, statistical variables (mean, standard deviation, maximum, minimum and median) are calculated within each TW (Li et al., 2015) as is described in Table III, i.e. feature number 6-80. The number of the top right corner of the feature is the length of TW, so ‘5’ in $mean_yawt5$ means 5 s length of TW and ‘4’ in $mean_yawt4$ represents 4 s length of TW, see feature # 6 and # 7 as examples.

Frequency-domain feature extraction has already been used in anomaly detection (Chen et al., 2015; Chandola et al., 2009). Fast Fourier transform (FFT) was used to transform time-domain signals into frequency-domain (Heckbert, 1995). The maximum value of FFT coefficients within TW is a good indicator to represent the property of frequency signals (Mörchen, 2003). The description of the frequency-domain features are listed in Table III, with feature number 81-95.

To evaluate extracted features, both LC and LK datasets should be labeled. Take LLC for example, as shown in Figure 4, the ego vehicle (blue) intends to overtake the slow vehicle (red) by left lane change. The moment that the left wheel of the ego car just crosses the central dotted line is marked as the initial LC time t₀. Based on the study in Salvucci and Liu (2002), normally drivers tend to start LC maneuver approximately 5 s before actual LC. Thus in this paper, time series between t₀ and 5 s before are labeled as LC behavior. To ensure LK data sets are separation of LC data sets, LK behavior are labeled between 10 and 15 s prior to t₀. It is the same way to label RLC data sets.

In the view of statistics, the p-value is commonly used to test whether there is statistical significance between two groups. In our case, if there is statistical significance between LC data sets and LK data sets, the extracted features are probably good indicators to classify LC and LK maneuvers. However, only using p-value to evaluate significance is insufficient (Sullivan and Feinn, 2012). The effect size, such as Cohen (1988), is also used as an important evaluation metric (Cohen, 1990):

where:

• d = Cohen’s index;

• M₁ = mean of the first group data;

• M₂ = mean of the second group data;

• S₁ = standard deviation of the first group data; and

• S₂ = standard deviation of the second group data.

To define significance level, Cohen defines the effect class as follow (Cohen, 1992):

• d < 0.5 = small effect;

• 0.5 ≤ d < 0.8 = medium effect; and

• d ≥ 0.8 = large effect.

For each LC maneuver, we label LC and LK data sets and calculate both Cohen’d and p-value for each feature. Then for all the LC cases, we average the Cohen’d and p-values to get the mean for each feature in each scenario.

To test if the selected features have advantages for machine learning techniques over all features, the SVM, NB, DT and KNN are chosen to evaluate classification performance. We then built the above learning-based models using the Statistics and Machine Learning Toolbox[¹]. Here, the SVM model was set with a Gaussian kernel function, and NB with Kernel smoothing density estimation method, DT with the default setting and KNN using empirical prior with k = 1. The datasets used for training the machine learning models are the same, which are labeled by the method presented in Section 4.3.

All the evaluation results (Cohen’ d and p-value) for each feature can be found in Table AI. A p-value smaller than 0.05 can be regarded as having statistical significance and a Cohen’s d value larger than 0.8 has large effect level (Cohen, 1992). By following this two criterion, we mark each feature with Cohen’ d larger than 0.8 and p-value smaller than 0.05 as red in Table AI. The red-marked features have great influence on the corresponding LC maneuver (LLC or RLC), and thus can be selected as strong features for LC maneuver recognition. Overall, based on the features marked as red, we find the following:

• Although some features (p < 0.05) have shown statistical significance (marked as blue), they have only medium or small effect size (Cohen’ d < 0.8). This result also coincides with that only using p-value to evaluate statistical significance is not enough (Sullivan and Feinn, 2012);

• Original features of yawRate_t (#1), az_t (#2) and ax_t (#3) and compound feature $TLCt−1$ (#5) are not strong features for LLC case with no items marked as red. For RLC, only az_t and $TLCt−1$ in RLC Scenario 0_1 can be regarded as strong features. This implies that the common empirical knowledge of using these features is not that much convincing.

• We mentioned that $TTCt−1$ is only calculated when the front cell of the ego vehicle is occupied by an object vehicle (Cell_m = 1). $TTCt−1$ is marked as a strong feature in the LLC case, which demonstrates that the potential of rear-end collision does influence drivers’ LC decision. In many research, a hypothesis – if the driver follows a leading vehicle which is too slow, he/she would probably maneuver a LC to overtake the slow leading vehicle – was made. This analysis from naturalistic driving data proves that this hypothesis is reasonable.

• Features #56-#60, which refer to mean_ax, are the least important features for LC maneuver recognition, with no item marked as red.

• To analyze the TW features (#6-#95), we take the marked strong features in LLC Scenario 0_0 and LLC Scenario 0_1 for instance. Here, we segment the Table vertically with 5 features in a group, e.g. features.

Features #6-#10 are related to the same feature mean_yaw but with different TW from 5 to 1 s, and so on. The detailed illustration can is shown in Table AI, where the features with the largest Cohen’d and the smallest p-value (marked with ‘▲’ and ‘▼’, respectively) demonstrate that they have the strongest effect on LC. From these peak and valley values we find that features with the largest Cohen’ d are also likely to have the smallest p-values, except for Feature #31 and #32 in LLC Scenario 0_0. We select the final features for each scenario based on the marked peak and valley features, and for the special case like feature #31 and #32, the features with large Cohen’d (e.g. #31) are selected.

Based on the marked features and results, the final selected features in each LC scenario are listed in Table IV. It can be found that different LC scenarios have different features sets. The number of selected features from all 95 features for each LC scenario ranges from 8 to 16. There is no feature eligible for all LC scenarios. Only using original features and compound features (#2, #4, #5) are far less enough, because no such kinds of features have been selected at all in LLC Scenario 0_0 and LLC Scenario 1_0, RLC Scenario 0_0, and RLC Scenario 1_0.