TerraFlow Performance

In order to investigate the performance of our programs we performed experiments with different main memory sizes on a set of real-life terrains of various characteristics.

Platform: We performed experiments on Digital Personal Workstations with 500 MHz Alpha processors and 766MB of main memory running FreeBSD 4.0. The workstations have local striped disk arrays consisting of 8GB 10000 RPM Cheetah and 36GB 10000 RPM IBM drives. For each experiment we rebooted the machine with the desired amount of main memory. For TerraFlow we set the TPIE main memory limit to 90, 200, 400 and 500~MB, in the experiments with 128, 256, 512 and 766~MB of main memory respectively, reserving the rest of the memory for the operating system. We experimented with relatively low main memory sizes in order to simulate the behavior of the programs on larger terrains.

Total running time of Terraflow at different main memory sizes. Data size is in million elements and running time is in hours.

128MB main memory

512MB main memory

TerraFlow completed succesfully on our biggest dataset, Washington State at 10m resolution (see terrains), which was made available by David Finlayson at the Mountain Drainage Basin Research Group, University of Washington. The whole grid contains 1066 x 10^6 elements (33454 by 31866), and the whole process takes 63 hours.

TerraFlow runs on ham, 1GB RAM, TPIE -m500M
Dataset Fill (-O3) Flow (-O3) Total
wwash 31:30 hr, out 31:23 hr, out 62:53 hr

This grid cannot be processed by ArcInfo! It dies with a fatal error in what looks like a file position overflow of a 2GB limit.

Detailed Terraflow Running Times

Fill and Flow are fully optimized (compiled with -O3, -NDEBUG).

128MB RAM (TPIE -m90M) runs:

Dataset Flow Fill Fill + Flow
KAWEAH 0:58 min, out 2:10 min, out 3:08 min
PUERTO RICO 1:19 min, out 7:31 min, out 8:50 min
SIERRA NEVADA 12:12 min, out 16:58 min, out 29:10 min
HAWAII 3:22 min, out 35:09 min, out 38:31 min
CUMBERLANDS 26:52 min, out 1:37 hr, out 2:04 hr
LOWER NEW ENGLAND 39:55 min, out 2:02 hr, out 2:42 hr
USADEM6 2:03 hr, out 6:40 hr, out 8:43 hr
USADEM2 6:43 hr, out 10:40 hr, out 17:23 hr

256MB RAM (TPIE -m200M) runs:

Dataset Fill Flow Fill + Flow
KAWEAH 2:11 min, out 0:59 min, out 3:10 min
PUERTO RICO 7:07 min, out 1:21 min, out 8:28 min
SIERRA NEVADA 16:17 min, out 11:47 min, out 28:04 min
HAWAII 34:30 min, out 3:55 min, out 38:25 min
CUMBERLANDS 1:33 hr, out 26:06 min, out 1:59 hr
LOWER NEW ENGLAND 2:01 hr, out 39:26 min, out 2:40 hr
USADEM6 6:41 hr, out 2:01 hr, out 8:42 hr
USADEM2 10:42 hr, out 6:13 hr, out 16:55 hr

512MB RAM (TPIE -m400M) runs:

Dataset Fill Flow Fill + Flow
KAWEAH 2:19 min, out 1:02 min, out 3:21 min
PUERTO RICO 7:04 min, out 1:23 min, out 8:27 min
SIERRA NEVADA 15:32 min, out 11:13 min, out 26:44 min
HAWAII 33:23 min, out 3:19 min, out 36:42 min
CUMBERLANDS 1:32 hr, out 25:52 min, out 1:58 hr
LOWER NEW ENGLAND 1:55 hr, out 38:43 min, out 2:34 hr
CAPDEM 4:53 hr, out 2:41 hr,out 7:30 hr
USADEM6 6:44 hr, out 2:02 hr, out 8:46 hr
USADEM2 10:21 hr, out 6:05 hr, out 16:26 hr

Terraflow Speedup over other GIS Software

We compare TerraFlow with: For the running times and charts see Speedup.


<laura@cs.duke.edu>
Last modified: Mon Jun 2 12:42:37 2003 by laura.