Figure 1 AIP
Control Box
Design Review Spring 1999
Abstract
The Aircraft Imaging Platform is a part of the ongoing Space Systems
and Operational Laboratory (SSOL) project. The current project of SSOL
is to take aerial photos of Ames so that the images can help to determine
the amount of light pollution in Ames. And the mission of AIP is to continue
design of the navigation software in Visual C++ for use in a small private
plane with the help of GPS technology. The software assists the pilot to
fly in a desired path and control the settings of the camera, for instance
the interval between each photo. AIP will integrate GPS data to overlay
on video input from a video camera.
Problem Statement, Background and Content
AIP has designed a navigational program that shows the pilot the specific checkpoints and the flight path on a laptop computer. The program informs the pilot how far the plane is from the checkpoint. The checkpoint is a three dimensional point in space that marks where a picture is to be taken. If the pilot misses a checkpoint, the program recalculates a flight path that will lead the pilot back to the missed checkpoint. The program is designed to insure that the pilot covers all the checkpoints and no checkpoint is repeated. It also informs the pilot when a particular checkpoint has been reached. In addition, the program controls the camera shutter release timing. When the plane reaches a checkpoint, the program activates the protocol for automated shutter release. A user interface was design to facilitate the pilot’s operation of the navigational and photography control.
With the current system it is difficult to determine were a specific
photo was taken due to the fact that the GPS coordinates are not on the
photograph. Therefore, a system for integrating GPS data and live video
recording will be designed. This recording can be cross-referenced with
the still photographs to help determine the location of the photograph
and other environmental variables that were present during the flight.
Design Objectives
Functional
The main functional objective of AIP is as follow:
End Product Description
The end product is composed of the AIP control box that runs in unison with a laptop computer, a camera, a video camera, a computer monitor, power supply and a VCR. The AIP control box and video camera run off of a DC battery, the laptop has it’s own battery and the VCR and monitor use a portable AC power source. The AIP box contains a Global Position System (GPS) receiver, a Differential Global Position System (DGPS) receiver, a Video Digital Overlay (VDO), a RS232 interface, a camera triggering system and voltage control circuitry. This hardware communicates with the laptop’s navigation and VDO controlling software through a serial and a parallel interface cable. Together they time camera triggers and display position coordinates on the video camera’s images that are recorded on the VCR.
The laptop communicates with the GPS-DGPS system through the COM1 port on the laptop and the RS232 interface circuit in the control box. This interface is capable of two-way communication that enables the laptop to receive GPS data and also run a diagnostic program that can change on-chip settings on the GPS board. Having this diagnostic software on the laptop can be an invaluable tool during field-testing, helping to confirm proper operation of the GPS board.
Communications with the VDO and the camera triggering system are done through the laptop’s parallel port. This allowed for the controlling of the camera triggering and communicate with the VDO without adding additional circuitry or connecting cables that would have increased complexity. The VDO and camera triggering signals are each given a dedicated data line on the parallel cable. The VDO receives serial style data that contains configuration data as well as the text characters to be displayed. The camera triggering is connected to a transistor switch inside the control box that controls the camera shutter.
The laptop is the user interface for this system. A person that desires to take aerial photos must first enter the longitudinal and latitudinal coordinates into the waypoint generating software on the laptop. This software converts the coordinates into a form the navigation software will understand and saves it to a file for later reference. When this is done the pilot can run the navigation software and VDO software on the laptop. The navigation software will read in the waypoints from the file and display them in three-dimensional space in order to help the pilot maneuver the aircraft over the correct positions to take photographs. When a position is reached, the camera will be triggered to take a photograph.
During the flight the VDO software receives GPS data in character format
and sends it to the VDO so the data can be overlaid on the video camera’s
images. This provides a reference from the images of the ground to the
GPS coordinates. The still photographs from the camera will then be compared
to the output from the VDO to help determine the exact coordinates at which
the pictures were taken.
Assumptions and Limitations
There are some inherent limitations with this project. Some of the limitations had to be dealt with are:
The AIP flight system is composed of both a software oriented user interface,
and a hardware oriented data acquisition system. Each of these components
operated largely independently, and as such were largely handled as separately
designed systems. The hardware system consists of the GPS receivers, VDO,
video and still cameras, monitor, and VCR. The software system consists
of a navigation and triggering software system, a waypoint entry program,
and a VDO communication program.
Technical Approach (hardware)
The main functions of the hardware system include the following:
Figure 1 AIP
Control Box
GPS-DGPS Receivers
To receive the GPS coordinates a GPS-DGPS system was selected by the AIP-1 team. The Motorola VP Oncore GPS board was selected as the standard GPS board. The board receives only the regular GPS signal, which is not highly accurate. To improve accuracy a Communication systems International SBX-2 differential GPS board was purchased. The differential GPS board receives a coordinating signal from Coast Guard towers that compensates for the minor errors associated with the standard GPS signal. This allows for highly accurate position information to be received.
This differential receiver is referred to as a beacon receiver. The SBX-2 is a 300kHz minimum shift keying demodulator. It features a two-channel fully automatic search algorithm. The corrections are broadcast from radio beacons under the authority of the International Association of Lighthouse Authorities. The nearest one is located at Dubuque Iowa. It can broadcast over almost the entire state of Iowa. There are a few counties in the extreme northwest corner that are not covered. The transmit pin on the SBX-2 is connected to the receive pin on the Motorola board. The output of the SBX-2 is in the RTCM 104 format. This is a standard interfacing format from DGPS to GPS receivers.
The output form the Motorola board is in the format of NMEA (National
Maritime Electronics Association) 0183. There are continual updates on
the web for this format. The GPGGA field was chosen for the AIP application.
This happens to be the same field type that is currently used for the SSOL
balloon flights. It gives time, position, and fix related data. The message
fields consist of ASCII strings that are broken apart to obtain the necessary
data.
The GPS board is connected to the Differential GPS board to ensure reliable GPS coordinates are received. The next problem is to route the GPS coordinates data to the
laptop computer for further processing, such as character parsing and formatting by
software running on the laptop. In order to do this, an additional RS232 chip is used
along with the serial port on laptop and the RS232 interface on the GPS board. GPS
coordinates coming in from the GPS receiver (GPS and DGPS) are sent to the laptop
through the serial communication channel COM1.
Still Camera
Still cameras will be necessary for most of the uses of the AIP system. So far one camera has been purchased for this role. The Pentaz ZX-50 ( figure 2) was selected due to its capability for nighttime photography. The camera will be connected to the triggering system with a remote triggering cord. A manual trigger will also be attached to
the system to allow for manual operation of the camera.
Figure 2 Pentax
ZX-50
The camera will be triggered to take a picture when the laptop determines that a waypoint has been reached. A data line connected from the parallel port of the computer is attached directly to the gate input of a BJT transistor that acts as a switch for the camera. The camera triggering cable is connected across the input and output of the transistor. When the triggering signal is sent to the BJT, the triggering cable from the camera is shorted and the camera takes a picture.
Video Camera
A black and white CCD pinhole camera will be used to provide the video image for the VDO. The camera will be mounted beside the still camera on the camera mount (figure 3). Its video output will be sent the VDO boards composite input. The camera uses a 12 volt 90mA DC power source. It is sensitive to .5 lux with an F-stop of 1.8, making it ideal for a wide range of uses including night flight.
Video Digital Overlay (VDO)
The VDO provides the capability to combine the video image received from the video camera with the GPS data that is received. To achieve this function the BOB-1 VDO board from Decade Engineering was chosen. BOB-1 can display 10 rows by 24 columns of text with a variety of other options. The exact specifications for the VDO will be included in the appendices. The VDO receives the video input signal from the video camera, connected though a standard RCA cord to the input lines of the board. This image is combined with data from the laptop and output to the VCR and monitor where it is displayed and recorded for future reference.
The data input to the VDO is simple TTL (5V logic) serial connection, using Clock, Data, and Select lines. The Clock signal will be sent from the Laptop across a data line on the parallel cable. The Select line will be tied to +5 Volts to keep the VDO activated, since the VDO will be needed as long as the system is activated. The data line accepts data as a serial string, which is sent from the laptop across another data line on the parallel cable.
The data sent to the VDO will consist of Latitude, Longitude, altitude, and the time. This will be used to help recreate a landscape from the pictures taken by the still camera. Also, this will help identify the specific GPS coordinates where the pictures were taken.
Monitor and VCR
A black and white Apple computer monitor and a standard VHS VCR are
used to record and display the combined video image from the VDO. A DC/AC
converter is being used to provide the power source for these two items.
Camera Mount
The still and video cameras are mounted to steady them during a flight. The current mount (figure 3) is a simple design with a resting slot cut for the still camera. Foam padding is used to rest the camera on to minimize vibration in flight. Also Velcro straps are attached to secure the camera. The video camera is attached at the top of this slot so both cameras will have a good perspective out of the camera hole. Four bolts are used to allow the mount to rest above the base of the plane and allow for adjustments so the mount can be leveled in flight.
Figure 3 AIP
Camera Mount and Video Camera
Technical Solution (Software)
The system completeness evolves through the use of commercial software products to resolve the following issues:
GPS Receiver Interface
The software that interfaces with the GPS receiver is located in pseudoinput.cpp. It retrieves the GPS data from serial port and sends it to requesting functions. The GPS data is received in NMEA format, essential a test string with data fields separated by commas. The receiver interface parses the latitude, longitude and altitude data from the string and makes it available for the rest of the program to use. The interface also saves this data to a file for later use by the VDO overlay software to combine the GPS data with the video camera image.
Laptop Navigation Aid
The Laptop Navigation Aid as shown in figure 4 was created using OpenGL for the graphics and Visual C++ for the non-graphics. The display in figure 4 is divided into four sections, statistical information, 3-d view and location of waypoints on flight path, top down view of plane and waypoints’ locations, and waypoint database management.
Statistical Information
This section contains information about the aircraft’s current altitude, speed, heading, longitude, latitude, and etc. The software that generates this display is located in the file statlist.cpp & statlist.h. The software receives its information directly from the GPS receiver during the flight.
Top Down View
This section contains a visual image of the aircraft and waypoints from the viewpoint of looking on top of the plane. It provides a graphical plane that shows the current position of the aircraft and a smoke trail to indicate previous travel. The software for this section is found in the Topdown2.cpp and accompanying header files.
Figure 4: Laptop Navigation System Display
3-D View
This section illustrates the flight path to be taken and the waypoints that are located on that path. The waypoints are represented as spherical balls with a line extending from the waypoint to its reference point. The waypoints are "flying" over a grid layout that is to represent the land below the waypoints
VDO Interface
The VDO system allows for a picture taken during the flight to be properly identified. The VDO system is comprised of live video recording that is overlaid with the GPS data as received from the GPS receiver during the flight. The interface software is interfaced with BOB-I (low power video character overlay device).
The interface software sends the GPS data to the VDO board across the parallel interface connected to the control box. The GPS receiver interface software stores the current GPS data in a file. This file is read in by the VDO interface software and sent to the VDO board in proper format. This data is then incorporated into the video image from the video camera and recorded on the VCR for later reference. The source code for the VDO interface is located in vdo.cpp
Camera Triggering System
The camera triggering system controls when the camera will take a photo of the land. It checks to position of the aircraft (via GPS data) to verify if the plane is within a valid distance of the next waypoint. If the aircraft is within the specified distance from the waypoint a signal is sent to the camera triggering system via the parallel port. The software for the camera triggering system is located in the file statlist.cpp.
Waypoint Database Management
Figure 5: Waypoint Database Management
The Laptop Navigation simulator also provides a waypoint database management tool. (see figure 5 below). The database was created to allow the pilot to enter the locations where pictures need to be taken before a flight actually begins. The latitude, longitude, and altitude of each waypoint is recorded and saved into a text file for later reference by the program. Once the flight has begun, the software will continuously reference the database to determine current navigation information to the next waypoint.
File Listing and Descriptions of the Entire Software System
|
|
|
| Pseudoinput.cpp | Retrieve GPS data from serial port and send to requesting functions |
| Pseudoinput.h | |
| Test.cpp | Functions for reading waypoints from file |
| Resource.h | |
|
|
|
| MainScreen.cpp | Main screen implementation file |
| MainScreen.h | |
| Graphics.cpp | Miscellaneous graphic functions (i.e. fog, colors, etc.) |
| Graphics.h | |
| Main.cpp | Begins setup and display for the main screen |
| 3dprim.cpp | This file generates the 3-d graphics of the
flight path and the location of the waypoints along the flight path that
are view in the main display window or the flight path simulator.
|
| 3dprim.h | |
| 3dview.cpp | This file is basically, the driver program for the 3-d generation of the flight path and waypoints that are viewed in the main window display. |
| 3dview.h | |
| Topdown1.cpp | Provides the functions for the class Topdown that creates the topdown view of the flight path and waypoints. |
| Topdown1.h | |
| OGLStatic.cpp | Base class for all OpenGL windows |
| OGLStatic.h | |
| Statlist.cpp | This file calculates the statistical values that are located in the statistical window of the Laptop Navigation System Window Display. It also contains the camera triggering software, and the system’s advancement to the next waypoint. |
| Statlist.h | |
| Waypoint.cpp | Handles the waypoints. It contains the data for the waypoints that is to be recorded. It reads the data from a file and output the number of time that data is recorded at that point to another file. |
| Waypoint.h | |
| WaypointDlg.cpp | Provides definition for the waypoint class |
| WaypointDlg.h | |
| Db.cpp | Implements the database that manages the waypoints |
| Db.h | |
Project Success
Overall, this semester has been successful. A full system test flight was preformed in March, however the test was unsuccessful. The GPS board was not initializing properly and therefore not receiving data. One more full system test flight will be done, but should be done by the time this report is presented. Most of the major milestones have been reached this semester.
Parallel port constraints with the windows programming environment prevented the integration of the VDO interface code directly into the main program. So instead a separate C++ program was written to run alongside the rest of the AIP code in a DOS window. Unfortunately several parts on the VDO board were been damaged and significant time has been spent on repairing the board, which has delayed many system tests. This has prevented the full testing and final integration of the VDO into the AIP system. The camera triggering system was finished. Rather than continuing to use the PIC microcontroller that was originally supposed to manage this task, a simple BJT transistor is being used to control camera triggering. This was done due to several problems encountered in using the PIC for this task. Work on the system board and circuit design was also completed. An additional parallel port was added to the system box for connecting the triggering device, and VDO board to the laptop.
Documentation for the system has not yet been done. The setbacks encountered
with the project this semester have delayed the documentation process.
Testing of the end product
Testing was started with the navigation software first. This was done by hooking the COM1 port of the laptop to a computer that was transmitting simulated GPS data. When the simulated route that the GPS data took passed through the waypoints the camera was successfully triggered. The VDO was then connected to the same system to see if it would overlay the correct coordinates to the video. The VDO failed to operate properly in the initial tests which lead to the development of a separately running dos program for interfacing with the VDO board.
Hooking up the GPS to the laptop to see if the GPS board was communicating to the laptop was the next portion of testing. During this initial test the DGPS antenna was unavailable so precise data was not expected. The interface with the software was working correctly except for the lost precession from not having the DGPS. Next the system was loaded into a car to test how it would operate in a moving enclosed vehicle. This test was also successful and didn’t reveal any problems.
During the initial tests the GPS initialized almost instantly and had
no problems receiving data within the enclosed car. Unfortunately this
didn’t happen during the first test flight. On this flight the GPS never
initialized and the triggering system was unable to be successfully tested.
However, the proper operation of the video camera was
confirmed. The hole out of the plane is small and in a well. Thus a redesign of the mount should allow the camera to sit deeper in the well to get a better view.
To find out why the GPS didn’t initialize during the flight it was hooked up to diagnostic software to make sure that it was operating in the right mode. Everything seemed to check out during this diagnostic test except the GPS never "locked on" to its location. Once the GPS was brought outside, where it had a clear view of the satellites, it eventually reinitialized and reported GPS coordinates again. Some more in-car testing was done. It was found that if the GPS antennas weren’t situated properly, within an enclosed vehicle, the GPS board wouldn’t have a clear enough signal to process its location. However, it appeared that if the antennas were close to the windows it could get a clear enough signal to operate correctly.
The final test flight has not yet been done due to weather constraints. The final test will be a combined final operation of the system and will fully test all components.
Weather permitting the final test flight will be conducted the week
of the 26th and the results of that flight will be appended
to the current report on the week of May 3rd.
Future Work
Since AIP is an ongoing project, there are many things that can be done
to improve the system. Some things that can be done in the future is as follows:
Human Effort Budget
| Personnel Estimated | Total Hours Estimated | Personnel Used | Total Hours Used | |
| Determining Needs | ||||
| Team Meetings |
6
|
120
|
6
|
90
|
| Meetings with users |
2
|
10
|
||
| Aquatinting new members with AIP |
4
|
80
|
4
|
40
|
| Preliminary Design | ||||
| Design new software features |
4
|
30
|
4
|
20
|
| Hardware design changes |
2
|
20
|
3
|
45
|
| Software Implementation | ||||
| Familiarizing with Visual C++ |
4
|
80
|
4
|
60
|
| Modify and debug code to add new
Features |
Not Estimated
|
Not estimated
|
4
|
70
|
| Parallel port programming and
Debugging |
2
|
20
|
2
|
20
|
| User Documentation |
2
|
20
|
2
|
13
|
| Code Documentation |
2
|
40
|
4
|
30
|
| Other Design | ||||
| Camera mount design and
Implementation |
Not Estimated
|
Not Estimated
|
1
|
5
|
| Hardware redesign and
Implementation |
2
|
20
|
2
|
50
|
| Testing | ||||
| Testing with simulated data |
4
|
10
|
4
|
40
|
| In flight testing |
4
|
20
|
4
|
10
|
| Total |
462
|
503
|
Changes in the human effort budget have largely resulted from inaccurate estimates in the amount of time expected to complete tasks. Most tasks were underestimated. This is a common problem in effort budgets, as it is hard to estimate the time it will take to complete tasks before they are began.
Final Spring 1999 Time Line
Monetary Budget
Spring 1999 Budget
| Item | Quantity | Estimated Cost | Actual Cost |
| Misc. Interfacing
(cabling, wires) |
Several | $50 | $30 |
| Hardware components
(circuit board and components) |
Several | $50 | |
| Poster | 1 | $50 | |
| Test Flights | 2 | $500 | $80+ second flight |
| Voltage Converter | 1 | $50 | Not needed |
| Camcorder | 1 | $200 | Not Needed |
| Miscellaneous | Several | $50 | $20 |
| Total | $950 | $130 |
Equipment provided in previous semesters
| Item | Cost |
| Imaging equipment mounting frame | $5.00 |
| Cable and connectors | $30.00 |
| Film Camera(1) | $400.00 |
| Video Camera (b/w) | $165.00 |
| Video Digital Overlay | $200.00 |
| PIC Microcontroller | $20.00 |
| AC/DC adapter (donated) | $0.00 |
| VCR | $175.00 |
| Monitor (donated) | $0.00 |
| DGPS receiver (donated) | $0.00 |
| Batteries (12v for TV/VCR, 9v for PIC, VDO and GPS) | $165.00 |
| 3200 speed test film (2 rolls) | $20.00 |
| Video tapes | $5.00 |
| Flying Hours (2 @ 60ea.) | $120.00 |
| Presentation and printing costs | $25.00 |
| Postal and phone expenses | $5.00 |
| Laptop Computer | 2870.00 |
| Development Tools (Visual C++) | $100.00 |
| Posters | $101.89 |
| Miscellaneous | $10.00 |
| Total | $4416.89 |
The AIP team benefits from funding provided by the Iowa Space Grant
Consortium, through the Space Systems Operational Laboratory. This provides
the team with the resources to complete the design objectives. Since this
is an ongoing project, much of the equipment has been purchased in previous
semesters. This semester's expenses centered on interfacing cables and
flight costs.
Lessons Learned
A project such as this always imparts a variety of lessons to its participants. One important thing learned this semester was the importance of not only teamwork but also organization. Teamwork this semester allowed large amounts of work to be done. The organization also allowed the team to keep on track, with both deadlines and goals in mind when working.
Test preparation is another important issue encountered this semester.
No matter how had a project has been worked on, getting beneficial data
from testing is no easy task. This team learned much about doing good tests,
but there is still much improvement that can be made for the future. Advance
preparation are perhaps most important. Since the most important tests
for the AIP system are in-flight test, which can not be held on a whim
due to cost, it is important to get the most out of every flight taken.
This requires much preparation and thought, so that important issues are
addressed before the flight and that there is not a waste of time in the
air.
Team Members
| James Kitson (Cpr E 482) Team Leader
312 Hilcrest Apt #3 Ames, IA 50014 Phone: (515)292-7078 E-mail: jkitson@iastate.edu |
Wai Chan (CprE 482)
PO Box 1248 Ames, IA 50014 Phone: (515)572-4781 E-mail: chanwai@iastate.edu |
| Mark Dietrich (EE 461)
Helser 2613 Louden Ames, IA 50012 Phone: (515) 572-2572 Email: mdietric@iastate.edu |
Anita Keely (CPRE 481)
Oak 3022 King Ames, IA 50013 Phone: (515) 296-6376 Email: akeely@iastate.edu |
| Clint Sanders (CPRE 481)
2300 Mortensen #6 Ames, IA 50014 Phone: (515) 268-9456 Email: csanders@iastate.edu |
Jason Sturtz (CPRE 481)
2101 Oakwood Rd. #301 Ames, IA 50014 Phone: (515) 296-2012 E-mail jsturtz@iastate.edu |
| Advisor | Client |
| Dr John P. Basart
333 Durham Iowa State University Phone: (515) 294-4955 E-mail: jpbasart@iastate.edu |
Iowa Space Grant Consortium
Iowa State University 408 Town Engineering Ames, IA 50011 Toll Free: (800)854-1667 Voice: (515)294-3106 Fax: (515)294-3262
|
Summary and Conclusion
The Aircraft Imaging Platform project has been very successful. This semester the team has flown multiple flights, and though problems have been encountered, they were overcome. The complex system that is AIP is now more functional then ever before, and is ready to begin the actual research that it was designed for. There is always more work that needs to be done in any project. Still, the achievements this semester have been remarkable, and almost two years after its inception the AIP project will begin to fulfill its mission of making research using aerial photography a much easier enterprise then before.