Establishing a robust testing approach for displacement measurement on a rotating horizontal-axis wind turbine

Health monitoring by conventional sensors like accelerometers or strain gauges becomes challenging for large rotating structures due to the issues with feasibility, sensing and data transmission. In addition, acceleration measurements have low capability of presenting very small frequencies, which happen very often for large structures (for instance, frequencies between 0.2 and 0.5 Hz in horizontal-axis wind turbines). By contrast, displacement measurement using stereo vision is rapid, non-contacting and distributed over the structure. The sensors are cheaper and more easily applied to many places on the object to be measured. Horizontal-axis wind turbines are one of the most important large rotating structures and need to be measured and monitored in time to prevent damage and failure, and the blade tip position is one of the key parameters to measure in order to prevent the blade hitting the turbine tower. This paper presents a clearly described and easily applicable procedure for measuring the displacement on the components of a rotating horizontal-axis wind turbine with stereophotogrammetry. Paper markers have been applied on the rotor and tower of a scaled-down horizontal-axis wind turbine model in the workshop and the displacement measurement method has been demonstrated by measuring displacement during operation. The method is mainly developed in two parts: (1) camera calibration and (2) tracking algorithm. We introduce an efficient camera calibration method for measurement in large fields of view, which has always been a challenge. This method is easy and practical and offers better accuracy compared with 2-D traditional camera calibration. The tracking algorithm also works successfully and is able to track the points during rotation within the measurement time. Finally, the accuracy analysis has been conducted and has shown better accuracy of the new calibration method compared with 2-D traditional camera calibration.


Introduction
Regarding the increasing industrial advances and the world's population growth, the fossil fuel sources will soon not meet the human need for energy. In such a situation wind energy is an environmentally friendly alternative that can decrease the 25 dependency on the declining fossil fuels (Herbert et al., 2007). Prospering the wind turbine technology has led to important concerns about the reliability of the energy production and wind turbine efficiency and reliable turbine operation also requires proper experimental tools and high quality testing methods to monitor the turbine behaviour.
Traditional contact transducers such as strain gauges and accelerometers have been used for vibration analysis, health monitoring and structural displacement of wind turbines (Larsen et al., 2002;Yang et al., 2014;Osgood et al., 2010 andHerlufsen et al., 2002) and other large structures like bridges (Hoffmann, 1989;Fukuda et al., 2013 andPark et al., 2005) but they have difficulties in measuring in large scale; the installation process that often includes wiring is costly and time consuming and the transducers load the structure with their weight that changes the dynamic properties of the structure and 5 need expensive correction (Ashory, 1999). The measured signal from conventional sensors such as accelerometers is not very accurate in measuring low frequencies of the structure to be studied (For instance frequencies between 0.2 and 0.5 Hz in horizontal axis wind turbines) and is including the centrifugal components (Najafi and Paulsen, 2017). In addition the contact transducers can only measure the structure in a few numbers of points and increasing the measurement points creates additional cost and complication (Hunt, 1998). As an alternative, non-contact optical measurement techniques provide faster 10 and cheaper possibility to measure displacement on large and rotating structures such as wind turbines. Stereo photometry is one of the common optical techniques for motion tracking of the objects that enables 3-D displacements measurements.
Stereo photometry or stereo vision estimates the 3D coordinate of the points using two or more 2D images taken from different angles. The preparation time is short and it could measure at many points on large structures.
The displacement of the turbine components (blades and tower) is an important parameter in analysing the rotor 15 performance and structural behaviour of the turbine during operation. Stereo photometry has been previously employed to estimate the strain and full displacement field of the turbine blades with digital image correlation (DIC) for investigation of the relative out of plane blade deflections (Winstroth, et al., 2014), rotor vibration measurement (Waren, et al., 2010a andPoozesh, et al., 2016) and blades damage detection (Leblanc, et al., 2013 andZarouchas andHemelrijck, 2014). DIC gives the continuous displacement distribution and is computationally expensive in monitoring large scale structures but 3D point 20 tracking (3DPT) which measures displacement in discrete points, is the preferable approach for outdoor and large scale experiments. In 3DPT, the optical targets, that can be simple paper markers (reflective or not reflective), are mounted at different places of the structure, as many as desired, and their 3D coordinates is tracked in time. 3DPT has been used for displacement measurements of the turbines in the recent years. The displacement measurement via 3DPT has been used to predict the full field dynamic strain of a model scale wind turbine blade (Baqersad, et al., 2015). However, stereo vision is 25 new in measuring vibration; it showed good agreement with conventional transducers like accelerometers and strain gauges in this filed (Warren, et al., 2010b andNajafi, et al., 2015). In, Najafi and Paulsen, 2017, 3DPT has been used to study structural vibrations of a model scale vertical axis wind turbine. Najafi and Paulsen, 2017, have investigated the challenges of using stereo vision for vibration analysis of complex geometries with sharp curvatures and out of plane components. In (Prowell, et al., 2011;Prowell, et al., 2012 andPaulsen, et al., 2012) the displacement measurements by point tracking stereo 30 photometry is used for structural response and modal properties of utility-scale horizontal axis wind turbines.
This study is focused on establishing a well-defined procedure to measure displacement on the components of a rotating horizontal axis wind turbine using stereo vision technique. A scaled down HAWT model, is used to demonstrate the displacement measurement method. Camera calibration is one of the main challenges in measurement in large field of views. Traditional way of calibration uses a calibration object with known and precise coordinates to calibrate the camera. This method is accurate and also efficient but it is unpractical for large field of view applications because of the calibration object size. There are also other techniques of calibration with no need to calibration object, they are called self-calibration. Self-5 calibration uses epipolar geometry of stereo pairs to reconstruct the 3D coordinates. These methods are flexible but the final results are not always precise and reliable because there are many parameters that need to be estimated (Medioni and Kang, 2005). In this study we updated the traditional calibration method for large field of views to be easier, faster and more practical. We also compared the results of the updated 3D calibration method with the traditional calibration that is conducted with a large grid in the background and the comparison shows the better accuracy of the new 3D calibration 10 procedure.

Experimental setup
The case study is a scaled down model of 3.6 MW Envision turbine (see Figure 1). The rotor diameter is 1640 mm and the blades are made from Aluminium with rectangular cross section of 5mm×8mm. The tower is also an Aluminium rod with the height of 1600mm and the cross section of 16mm in diameter. The rotational speed of rotor is between 0-150 rpm. A wire is 15 twisted around the individual blades to prevent vibrations induced by vortex that has formed behind the rectangular shape of the blade. Nikon lenses. The maximum frame rate of the cameras is 187 frames per second (fps) for full resolution images. The image acquisition system has limited capacity; therefore the longest acquisition time of stereo system with maximum frame rate (187 fps) is about 16 seconds. However with decreasing the frame rate the acquisition time increases, for instance the motion of the turbine can be tracked for about 150 seconds with 20 fps. Nevertheless with upgrading the acquisition system the 5 measurement time can be modified.
In this experiment the cameras are place about 7.5 meters away from the turbine while they are apart by about 3.5 meter.
This setup satisfies the rule of thumb which says the distance between the cameras should be between 3 and 1/3 time of the distance between the cameras and object. The simplest marker used in stereo vision is circular shape with a good contrast with the background. According to (Ozbek and Rixen, 2013) curvatures of the structure where the markers is applied, lead to perspective errors that needs to be corrected. In horizontal axis wind turbines, some parts of the blades are curved or deformed due to the loading during the rotation, this deformation and also the relative angles between camera and turbine causes the changes in the shapes of the 15 marker in the image from circle to the ellipse and the corresponding error correction is quite challenging due to the unknown Wind Energ. Sci. Discuss., https://doi.org/10.5194/wes-2017-49 Manuscript under review for journal Wind Energ. Sci. Discussion started: 27 November 2017 c Author(s) 2017. CC BY 4.0 License. instant blade deformation. To avoid these difficulties, we have changed the marker shape to the following shape, shown in Figure 3, with the diameter of 4 cm: Camera calibration is the estimation of intrinsic and extrinsic parameters of the cameras to relate the 3D coordinates of the word to 2D coordinate of the image. Extrinsic parameters define the location and orientation of the camera that contain the translation vector and rotation matrix. Intrinsic parameters describe the optical, geometrical and digital characteristics of the camera such as focal length, image centre in pixel coordinates, the effective pixel size in the horizontal and vertical directions, and, the radial distortion coefficient. Traditional camera calibration is the most common way of calibrating 10 cameras and has been studied and improved during years by different researchers (Tsai, 1987;Weng, et al., 1992 andZhang, 2000). This method uses a calibration object including a number of points with known coordinate to estimate the camera parameters. The traditional calibration is divided to 3 methods based on the calibration object dimensions: 3D, 2D (planar) and 1D (linear) calibration (Medioni and Kang, 2005). 3D calibration can be conducted very efficiently with very high precision but it requires expensive equipment and elaborate setup in the traditional calibration procedure (Faugeras, 1993), 15 but 2D traditional calibration is easier and less expensive (Sun and Cooperstock, 2005).
In this section two calibration methods are applied: at first the cameras are calibrated using a 2D calibration board with known coordinates and in the second part a new 3D calibration approach is introduced.

2D traditional camera calibration
The 2D traditional calibration has been conducted with a 2D printed grid applied on a wooden board. The dimensions of the 20 grid are 2m×1.9m and it contains 21 horizontal and 20 vertical lines that intersect at 420 points with known coordinates that are 10 cm apart in horizontal or vertical direction.

3D updated camera calibration with Leica
Traditional 3D calibration method can be conducted very accurately, as it has been referred at the beginning of this section, but it needs expensive and elaborate setup and expensive equipment such as two or three orthogonal planes. Setting up the traditional 3D calibration in large field of views, like full scale wind turbine experiment, is not feasible due to the difficulties in providing a precise calibration object. In our new calibration method, instead of using a huge 3D calibration object, we 15 used the markers that are applied on the turbine for the sake of measurement, as the calibration points. A Leica surveillance (Leica Nova MS50) is our smart device to determine the exact position of the points quite accurately and then the coordinate are used for 3D calibration of each camera. The Leica accuracy in x and y directions is dependent on the distance between the Leica and the object, hence the accuracy in x and y directions in the current case is about 0.0349mm. In addition the Leica accuracy in depth (z direction) is 1mm for measuring on reflective surfaces. 20 During the calibration the rotor was rotated (manually) by a specific angle step by step, in order to cover the whole rotor area by the calibration points and establish the collection of known coordinates for camera calibration.  The angel between the lines that connect the 5 ellipse center to the first and second picked marker coordinates on the ellipse in step 1-(black dots in Figure 6) is calculated and by knowing the number of image sequences between the first and second point (N1) on the ellipse, the first guess for the angular deflection between image sequences is obtained. As the turbine rotational speed is not fully constant during rotation, we need to update the angular deflection in each sequence. is calculated using the Harris-Stephens corner detection algorithm in the window. This algorithm is described in Harris and Stephens, 1988. 5-By knowing the exact position of the marker, the exact angular deflection is calculated (Δθ) 6-The approximate marker position in the next sequence is estimated using the angular deflection in the previous sequence. On the other hand the initial guess for the angular deflection in each time step is the angular deflection of 20 the previous sequence (Δθ 0 = Δθ) 7-The exact position of the marker is estimated by repeating the algorithm from step 3-to 5-.

Results and discussion
After calibrating the cameras by traditional and updated calibration methods, the 2D position of the markers in the image is found during the rotation using the tracking algorithm. The markers that are followed in time are shown in Figure 8:  It can be seen in Figure 9 that there is an offset, especially in depth direction, between the displacement obtained by the 5 traditional and updated calibration methods; this offset is due to having different origin of the coordinate systems in two different calibrations. There are also other minor differences between traditional 2D and new 3D calibrations (is more obvious in in-plane positions) that a part of it can be due to the slightly different direction of the coordinate axes.
For comparing calibration methods two different indicators are investigated: 1-the distance between the light rays at the intersection position 2-the distance between two markers during rotation. 10 The light rays from the marker to the cameras do not exactly intersect in the space due to the inaccuracies and the 3D position is regarded as the point with minimum distance from two rays (Trucco and Verri, 1998).
In the following table the averaged and maximum value of the distance between the light rays during rotation for point number 6 are listed in different rotational speeds. The photography sampling frequency is 50 frames per seconds and the recording time is 1 minute. 15   Table 1that the distance between the rays is much larger in 2D calibration using the grid compared to the 3D calibration using Leica. This comparison proves that the new calibration with Leica is more accurate than the traditional 2D calibration even with much fewer calibration points. The distances between the light rays in measurements with different numbers of the 3D points are relatively close but the measurement of the 3D calibration with 103 points on the turbine, shows slightly better result. It can be seen in Table 1 that by adding the background calibration points the distance between 5 the rays does not change significantly. This is a good and practically relevant point for full scale turbine measurement that shows having other calibration points than the points on the turbine is not necessary and does not improve the displacement measurement quality. It is expected to see better results with more calibration points; however 3D calibration with one third of the turbine calibration points looks still acceptable. This effect of the number of the calibration points on the calibration quality is an important parameter that also needs to be checked in the full scale experiment. 10 According to (Ozbek and Rixen, 2013), distance between the target points for a real turbine remains constant during rotation. In the present study, the blades of the model turbine are pretty flexible; hence the normal distance between the markers might change a little at higher rotational speeds such as 30 rpm especially at the blade tip region. Therefore the blade elongation is studied for rotational speed of 5 rpm as an indicator for calibration precision. In Figure 10 the change of the distance between markers 1 and 2 on blade 1 that are about 146 mm apart and markers 1 and 2 on blade 3 that are about 15 145 mm apart are plotted for the different calibrations approaches: The larger changes of distance between the markers with the 2D calibration could notify the less accuracy in this calibration compared to the 3D calibrations with Leica.

It is obvious in
The distance between two markers changes periodically in Figure 10, no matter which calibration method is used to obtain the displacement. The distance between the other two markers at almost the same area of the blade (markers 1 and 2) are also checked for the sake of reliability and repeatability and they also revealed the periodic behaviour of the elongation 5 between markers that is due to the repeating errors during operation. To investigate this behaviour, the spectrum of the distance measured with 3D calibration has been plotted in Figure 11: Figure 11: Power spectrum of the distance changes between points 1 & 2 on blade 3 calculated by 3D calibration using turbine points Figure 11 shows that the periodic behaviour of the elongation between markers is dominant by 1P and 2 P, where P is the 10 rotor rotational frequency (5 rpm ≈ 0.083 Hz). This could be because of the physical problems such as the light reflection, calibration inaccuracies, geometry miscalculations and also camera un-synchronization.
By looking at the pictures, it can be seen that each marker in Figure 10 is exposed to strong reflection once during one cycle that will be two times for a pair of markers, this could explain the 2P peak in the spectrum. For decreasing the unwanted reflections during displacement measurement, the markers can be printed with matt coating. 15 The geometry miscalculation which is mainly happening for circular markers is less likely happening in the current study. On the other hand the image of circular markers changes to ellipse in the image due to the blade loading and deformation during rotation and relative angle between camera and turbine. To calculate the center of the circular marker from its elliptical image, the knowledge about the exact angle of the rotor plane is required but in the current case no center calculation is needed due to the specific shape of that marker (Figure 3). 20 In the current study there is no external trigger or switch to start the cameras and they are triggered at the same time using software trigger (LabVIEW code) that could disturb the perfect synchronization between the cameras. This also could be one of the sources that makes the P peak occurred in the spectrum. The effect of the camera un-synchronization would be obviously more pronounced in full scale experiments with very larger dimensions and higher rotational speeds; therefore the image acquisition system should be equipped with the external trigger during full scale turbine testing. 25 The uncertainty of the measured displacement signal obtained with 103 point calibration is calculated using the generalized method based on the law of error propagation in a linear camera model of a stereo vision system. In this method, which is very well described in (Chen, et al., 2008), the uncertainty propagation in stereo reconstruction is explained in two main stages: camera calibrations and 3D triangulation to obtain the 3D coordinates from 2D projections in the images. In It can be seen in Figure 12 that the uncertainty values do not change significantly from root to the tip of the blade that show that the lens distortion is really negligible. 10

Conclusion
In this paper, a well-defined stereo photometry approach is introduced to measure the displacement of a rotating horizontal axis wind turbine. Camera calibration and marker tracking algorithm are two topics that are studied in this work.
Camera calibration for large field of views normally involves a lot of challenges. The traditional way of calibration that uses a calibration grid with known points is impractical for large scale uses, with a modern size wind turbine the calibration grid 15 would at least need to be 120x120 meters, and other calibration techniques such as self-calibration are not always precise and reliable. In this study a new 3D calibration method has been developed, the method is especially suitable for large field of views; in this study an operating horizontal axis wind turbines. The new 3D calibration method is easier, faster and has the big advantage that it avoids the use of a calibration grid. Instead of the calibration grid the measurement points themselves are used as the calibration points by determining their coordinates. A Leica surveillance device with a laser range finder is used in the calibration process to determine the exact position (x,y,z) of the calibration points while the turbine is at standstill. 5 During the calibration the rotor is rotated with a specific angle step by step, to cover the rotor plane of the turbine and establish the collection of known calibration points. The coordinates extracted by Leica are then used for a 3D calibration of each camera using a defined procedure. The markers are then used as 3D sensors of stereo photometry after the calibration and the cameras record their motion to archive the displacements by post processing of the images.
A comparison between the results from the new 3D calibration method and the traditional 2D calibration that is using the 10 calibration grid shows a higher accuracy using the new 3D calibration procedure.
The new 3D calibration procedure was then conducted using different numbers of the calibration points on the turbine components and also on the background. It is concluded that the background points are not necessary and didn't improve the calibration quality, this is very important in the full scale experiment as is would be problematic to install background points on a modern size wind turbine. The investigations also showed that decreasing the calibration points until 35 points on the 15 turbine for the imaging area of 2m×2m, still gives acceptable quality on the 3D calibration.To measure the displacement of the markers during the rotation, a tracking algorithm has been developed based on the circular motion of the rotor markers and the robust corner detection image processing algorithms for determination of the maker's position. This algorithm that keeps tracking the markers robustly during operation, updates its parameters based on the angular deflection of the marker in the last time step and the elliptical path of the markers in the images during the rotation. 20 Light reflection and camera un-synchronization are discussed as the main sources of the error during measurement that can be addressed using matt markers and also external trigger for the cameras in full scale experiments.