Work LogDavid Claus – Hilary Term 2003

Pending Work

GPS

  • Have plotted initial test data in Excel, but conversion from degrees to metres is suspect. This should be done in Matlab using code found online.
  • Need to record test data on football pitch to determine ability to track a line, drift etc. Put Seres and laptop on cart.
  • Determine what smoothing (if any) is performed within the Seres.
  • Determine improvements based on EGNOS corrections
  • Increase sample frequency to 5Hz
  • Record and process raw data

Target Location

  • Write up recent changes to algorithm, produce sample plots to justify additional stages.
  • Determine angle at which predictive region is required.
  • Work on detecting small target. Broaden pixel probe match criteria to detect small dots where the majority of the pixels are lighter.
  • Try other targets
  • Shoot another sequence from a distance, but one that does not include a pattern that matches the dots.
  • Shoot sequence outdoors to increase available light and possibly reduce motion blur. Also, GPS data will require outdoor footage.

Camera Location

  • Compute camera location from calculated centres.

Documentation

  • Edit and post GPS overview
  • Re-organise website so that edge detection is an aside. Look into using JavaScript to streamline buttons that link items.

Week -1

8 January 2003

  • Writing log for work done at the end of December.
  • Met with Andrew: subsample output image by scaling up, inserting warped image and then scaling back down; try weighted centroid calculation for blurred dots.
  • Scaling up requires multiplying by diag(3 3 1) as Andrew suggested.
  • On resizing the output image back down to the regular size it is important to use something other than ‘nearest’ interpolation otherwise nothing is gained. This doesn’t actually entirely solve the problem; it merely triples the resolution achieved.

9 January 2003

  • Interpolating image as it is inserted into the video. Need to calculate the size of the image area to interpolate, as the warped image is now too big. Edge of interpolation is taking in black values and resulting in a grey border to the inserted portion.
  • Scaled up a slightly oversized region of the video, and then trim the outer edges before re-inserting to eliminate the border.
  • The interpolated insertion is much steadier than the first attempt.
  • Wrote code to compute the weighted centroid for a dot. Also computing the centre by averaging the max and min pixels in each direction. Outputting these values and the computed centroid for the top left dot in each frame of target2.avi.
  • Plotting the centres as computed by each of the above three methods. The centroid and the weighted centroid follow each other very closely, but the max/min method is quite jittery and wanders.
  • While trying to compute test data for frames 500-525 of target4.avi found that current search algorithm failed. This was due to searching three points for the pixel probe (above, below and to the right). By eliminating the third point the algorithm was repaired.
  • On the target2.avi sequence the most deviation between centroids is 2 or 3 pixels. For the extreme zoom portion of target4.avi (frames 500-525) the most deviation was ~0.2 pixels.
  • From this it is concluded that there is no point in the extra expense to compute the weighted centroids.

10 January 2003

  • Sorted out Canadian tax issues.
  • Read Seres manual looking for possible explanation of data smoothing. Couldn’t find anything specific, but did locate a number of commands for getting data from the unit. I should probably start working with the raw data to see what is possible.
  • Went to check out the sports ground, but the porters weren’t there and I could not find the grounds manager. The field looks like it will be ideal. There is a wheelbarrow that will suffice to put the computer and Seres in, and the small amount of flooding on the eastern edge won’t interfere. There are goalposts for both American football (or is it rugby) and soccer.
  • Also found a cellular repeating tower off Marston Ferry Road that may be useful. It is BT Cellular’s, and they probably have engineering drawings for it as well as an accurate survey location. This may prove useful in the future as a sample structure for testing a reconstruction’s accuracy.
  • Re-working the web site. Motion blur can produce ellipses that may cause an ambiguity: a high vertical blur of small dots appears the same as a larger set of dots viewed from the side.

Week 0

13 January 2003

  • Checked with Mark and Paul of the New College Sports Grounds regarding use of the field for GPS calibration. They said it would be fine, and I could use their wheelbarrow. The large football field is supposed to be 95m between goal lines, and the penalty box is 18m wide.
  • Produced output video of target mapped to fronto-parallel plane. Centres computed from edge detection seem to be more accurate.

14 January 2003

  • Reproducing output videos of fronto-parallel mapping that includes all 200 frames for each data set. Also separated edge detected data from the searched data. Computed search data for first 200 frames. Found that centres computed from edge detection are not in fact more accurate. Yesterday I had results from the edge detection and the searching backwards.
  • The following frames are misaligned in the searched coordinates. Frame 378 returned 5 features, but the computed coordinates were accurate. The original video and computed coordinates should be reviewed to determine why this is failing.

Out a fair bit: 99, 201-203, 216

Out a little bit: 84-85, 118-119

  • Produced match plot for three pixel probe. Using VRMLplot from Deep Submersion Lab to create VRML output, and Cosmo Player to display it. The scan for the matches uses a fixed spacing in both the vertical and horizontal directions. At present I have only produced output for the target2.avi sequence as it has fairly constant dot dimensions. To scan the other sequence would require changing the spacing as things move along. Also, should produce a scan of every pixel within the dots, not just the pixels in the centres.
  • Will start in on computing 3D pose

15 January 2003

  • Printed out new target and taped to cardboard background
  • Ordered GPS rechargeable power supply

Thank you, your payment was successful
Merchant's Reference: DC13BN10000018
WorldPay Transaction ID: 31575280

  • Computing 3D pose. Used data from David Murray’s computer vision assignment to verify that I had the right algorithm. I can map points from that set-up correctly (or so it seems). When I revert to computing the pose for the actual video I get a little error for the first and third dot, and a fair bit of error for the second and fourth dots. The relative magnitudes of these errors are fairly constant over multiple frames.
  • The computation of the P matrix is independent of the K matrix. Currently K is the identity.
  • Tried recording new target video sequence; camera would not accept tape. Took tape into Jessops, played fine in new camera.

16 January 2003

  • After changing the position within the tape it played back fine in the camera. Hopefully it was just that it got stuck somehow.
  • Soldered fittings on to GPS power leads and lantern battery pack. Powered up fine with the battery, but the fuse blew when connected to the AC adapter. Not sure if this is due to a surge, or reversed polarity. Should connect it all up then turn on the wall switch, rather than plug in to a live connector like I just did.
  • Inquired at Clerk of Works office at New College regarding a plan drawing of the college grounds. Talked to the Deputy Clerk, who said David, the actual Clerk should be back around 2pm.
  • Power pack arrived from T-Leisure.
  • Talked with David Rolfe, New College clerk of works. He has some small plots of the ordnance survey for the New College area. There are survey markers outside the old college gate in Queens lane, and also at the corner by the King’s Arms. He uses these whenever he has to shoot any elevations around the college. Recommended that I check at Blackwell’s for any maps that I want. All the drawings he has are scaled, but don’t have any noted dimensions. I will give him a call and set up a time to meet and discuss the methods behind the British Ordnance survey.
  • Blackwell’s can supply a 200m by 200m plot for £25 (I think). They can furnish any area that we request. These could be useful for comparing results of the GPS/video system with a known standard. The price for a digital copy of the same size is roughly £40, while 160,000 m2 would be roughly £150 with VAT. This seems a little pricey, so for now I will try and get copies of David’s prints.
  • Looked around in town for replacement fuses, to no avail.

17 January 2003

  • Read latest issue of Computer Vision in the library. There is an article on computing camera and perspective from a plane by Carlesson and R?.
  • Reversed polarity on AC adapter power connector. Obtained quick-blow 1 amp fuses from engineering stores. Soldered leads for cigarette lighter adapter; this will allow connection to the power pack.
  • Tried logging GPS data on the New College sports grounds. Picked a miserable day for it! Placed computer and Seres in wheelbarrow and wheeled them around the lines on the field. Unfortunately the connection to the laptop was lost after once around the field. The data from that pass showed the whole range of HDOP values, which is discouraging. Some of the points are nicely linear, while there are still a fair number of points that are clearly in the wrong place. Some of these even have HDOP of 2.
  • Wrote code to rotate the GPS data points so they are aligned with the coordinate axes.
  • Target5.avi: plotted output of pixel probe for target area only. Found that increasing the tolerance of the probe so that more black values are allowed would improve results. Was able to get this frame tracked with a four point probe.

Week 1

20 January 2003

  • Spent the morning cancelling my registration for the Rhodes Centenary Conference in South Africa.
  • Attended first lecture of Scientific Computing II. I think that I might take this course; even though it isn’t all that applicable to computer vision per se it is relevant to the larger field of engineering, and may fill in a gap in my undergraduate math.
  • Met with Andrew. Next week we will meet at 11:00 on Tuesday as he is teaching a class with undergraduates.

21 January 2003

  • Read through latest issue of GPS World – contains articles on antenna interference and jamming.
  • Tried getting the new version of Matlab to work with the image processing toolbox, to no avail. I can’t get images to display in black and white. After retouching the rgb2gray function the images come in as negatives.
  • Working on sorting out the 3D pose calculations. So far I can get it for a test data set, and can normalize that data.

function [K,R,t,P] = testpose

rows = 720;

cols = 576;

% the world points

v = 113;

X=[0,v,v,0];

Y=[0,0,v,v];

Z=[0,0,0,0];

% the image points

x = [200,225,225,200];

y = [50,50,100,100];

%x = [208,255,204,252];

%y = [203,202,292,299];

% calibrate the data

C = [(rows + cols)/2 0 rows/2;

0 (rows + cols)/2 cols/2;

0 0 1];

calpts = inv(C)*[x' y' ones(4,1)]';

r = calpts(1,:)';

c = calpts(2,:)';

g = X';

h = Y';

% calculate the homography

[H,Hi] = cal(r,c,g,h);

sy = fix(min(c));

sx = fix(min(r));

K = [1 0 2;0 2 1;0 0 1];

H1 = inv(K)*H;

len = sqrt(norm(H1(:,1))*norm(H1(:,2)));

% len = (norm(H1(:,1)) + norm(H1(:,2)))/2;

r1 = H1(:,1)/norm(H1(:,1));

r2 = H1(:,2)/norm(H1(:,2));

t = H1(:,3)/len;

r3 = cross(r1,r2);

R = [r1 r2 r3]

t

P = C*K*[R t];

  • The relationship between the world coordinates and the image points is given by:
  • where P is the projection matrix. Substituting the z value for the plane and breaking P into rotation and translation components gives:
  • The rotation matrix consists of three columns, but since the z value in the world coordinates is always zero this can be reduced to:
  • The first two columns of the rotation matrix R must be perpendicular. If this is the case the third column will have the correct orientation. The first two are divided by their lengths (which should be the same, but may not be). Then they will be of unit length, as will the third column. However, this only works in a perfect world. The actual image data will not produce such well laid out vectors. As a result, the directions need to be massaged into the proper form.
  • The first two columns are coming in parallel if I use a K that is of the form that incorporates the aspect ratio, focal length etc. I don’t know why.

22 January 2003

  • Altered dataPose to account for change from image coordinates to the standard Cartesian layout. This improved the lengths of the two first columns. The calibration step is still producing very haywire data.

24 January 2003

  • Acquired more GPS data, which is more precise than that obtained previously. Weather was good; no clouds in sky, very little wind. First time around field (per2.txt) no SBAS, data wandered as it had previously. Approach to the field (approach.txt) was steady. Second pass around half the field, in the reverse direction with SBAS (half1.txt) was very steady. Third sequence was around the entire field forward with SBAS (perw.txt). The final data set was to go around half the field in reverse, without SBAS (half2.txt).

25 January 2003

  • Worked on assignment for Scientific Computing.

Week 2

27 January 2003

  • Updated web pages for computing camera position.

28 January 2003

  • Made cardboard cube for computing camera calibration matrix. Shot a very short video sequence of the cube.
  • Assembled match plots for various frames of the target5.avi sequence. Completed updates to web site.
  • Met with Andrew; sorted out problems with camera position codes. Will produce library of images of two types: positive target matches and mismatches.
  • Revised testpose and dataPose so they produce the correct output. Will need to get better estimate of K before proceeding. Need to determine camera location from projection matrix – see Hartley & Zisserman. Will also need to develop non-linear optimization to compute best possible R and t for each frame. Will require an examination of VRML to produce output animation showing camera viewing scene.
  • Writing code to extract target features from video sequence. Only deal with the largest feature within a selected area.

29 January 2003

  • Shot video in various locations in and around New College. Transferred video sequences to laptop, without compression. Tried several compression codecs to try and see if any had better performance in terms of compression artifacts. Didn’t find any that worked much better than Indeo 5.10, especially given the much smaller file size. Will try and work with the uncompressed video, and see how that goes.
  • Began work on second assignment for scientific computing.
  • Dropped video camera off with Andrew.
  • Wrote code to automate extraction of a dot from a frame, changing frames and videos, and logging the particulars of each image.

30 January 2003

  • Worked on assignment for Estimation course.
  • Writing code to extract samples of dots from video sequences.

31 January 2003

  • Read Ian Reid’s notes for reading group.
  • Completed sample extraction code.

Week 3

3 February 2003

  • Updated web page with samples of dots.
  • Plotted GPS data and posted to web page.
  • Met with Andrew.
  • Began looking into VRML tutorial.

4 February 2003

  • Created model of the wall and shelves in VRML.
  • Began computing camera location and rotation – setting viewpoint in VRML to correspond. However, this isn’t working satisfactorily yet.

5 February 2003

  • Set up new printer on lab computer.
  • Worked on assignment for scientific computing.
  • Began statistical analysis of target sample images.

6 February 2003

  • Continued statistical analysis of target samples. Will subtract mean, divide by standard deviation of each image, and then add on an offset of 230 and divide through by a standard deviation of 95. Otherwise the image is centred around the origin, with no spread.

7 February 2003

  • Completed statistical analysis, and split positive image set into training data and a sample set. Produced html to display these from within Matlab.
  • Now need to get negative samples.

Week 4

10 February 2003

  • Extracted negative target samples. Unfortunately, they were recorded in grayscale. Wrote code to parse ‘matches.txt’ and got colour versions; also saved grayscale and normalized versions. This will prove useful for scanning with different pixel probes.
  • Completing Scientific Computing II Assignment 2.
  • To determine the image normalization that would best separate the target samples from the non-target samples (positives from the negatives) a plot of the probe values for each class of sample was produced.

  • the blue values are the positives, red are negative.
  • Now compute the matches for this data set. [graph removed] The first columns are the correct positive identifications and the errors. The second set is for the negative samples. This data is suspect, as I couldn’t get the plots to change no matter what I did. Will look into this tomorrow. [ Turns out that I was computing the norm based on the ith entry in the matrix, not on the ith row.

11 February 2003