2009-06-01

Add a column to a text file from a raster

Text files are great

I have mixed feelings about software interoperability. Sure GRASS interfaces with R, and R can interface with Excel and Access if you have the ODBC drivers. But, often users do not have the energy to learn the best techniques for moving data back and forth between programs. Further, there always seems to be some quirk with data that prevents the transfer from going as smoothly as the user's manual examples do. Then, I remember the lowly text file. Whether you use comma separated values (csv) files or tab-delimited files, every software platform on the planet can read text files of records (lines) and fields (values on a line separated by commas,tabs or some other character). If you're trying to send data to someone else, you can be confident that your recipient can read them. It may not be the flashiest or fastest way to move data between people and systems, but it always works. Finally, when you use plain text files for data, you have many more options for working with the files. Tools such as bash, awk, sed, perl, and any other programming language you can think of will can be brought to bear on data analysis problems.

Attaching raster data to points in GIS

Switch gears for a moment from a common file format to a common analysis task. For researchers who work with spatial point patterns (like plots), a regular step in data analysis is to pull data from raster layers where they intersect the point data and add it as columns to the point data. Informally, these are known as "drill-down" operations. You have a stack of maps and you want to drill down through them vertically at point locations, grab the data, and add it to the points. This is an a relatively simple task in ArcGIS if you use Hawth's tools. It's also pretty easy in GRASS, though it involves a couple of steps. The function v.what.rast will put raster values into a column of a point dataset. The column must already exist in the dataset, which can be done through the command v.db.addcol and v.what.rast function just fills in the missing values with values where the raster intersects the points.

Adding raster values to text files

Now, back to text files. I recently worked with a graduate student who wanted elevation values from a lidar-derived digital elevation model (dem) on 4 samples totalling about 1100 plots. She is not a GRASS user, and so she sent me the locations of all the plots along with an identifier in csv format. I wanted to send her back the same csv files with elevation simply added as a field. I wrote a bash script to accomplish this quickly and repeatably for her files. In this script, all the GRASS functions are hidden from the user. Users simply input a csv file and the raster column you wish to add, and a csv file is output with that column added. The script requires the awk script that I posted previously, which recognizes the types of varibles present in the input csv. I have included a portion of the full script below. GRASS has a very particaular code style that it uses for running scripts natively (See g.parser for details). This makes the script very long. So, I've cut out the not-so-important parts. If you would like the full file you can get it here.

v.ascii.addrastcol

#!/bin/sh
##################################################################
#
# MODULE:       v.ascii.addrastcol
# AUTHOR(S):    R. Todd Jobe 
# PURPOSE:      Add a raster value field to a csv of points
# COPYRIGHT:    (C) 2009 by R. Todd Jobe
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
##################################################################
# . . . Grass header information removed
# . . . The cut code creates variables GIS_OPT_* and GIS_FLAG_*
# . . . Get the full code at:
# . . . http://www.unc.edu/~toddjobe/src/v.ascii.addrastcol
##################################################################
# Need space-delimited arguments for looping
GIS_OPT_INPUT=${GIS_OPT_INPUT//,/ }

# test for input raster map
eval `g.findfile element=cell file="$GIS_OPT_RAST"`
if [ ! "$file" ] ; then
   g.message -e "Raster map <$GIS_OPT_RAST> not found in mapset search path"
   exit 1
fi

for i in $GIS_OPT_INPUT; do
   # Check that input files exist
   if [ ! "$i" ] ; then
 echo "file $i does not exist" >&2
 exit $E_NOTFOUND
   fi

   # Create header line if necessary and get variable types
   if [ $GIS_FLAG_H -eq 1 ]; then
 skip=1
   else
 skip=0
   fi

   # Intermediate variables
   map=`basename $i .csv`
   tmp=`tempfile -s ".csv"`
   of="${map}${GIS_OPT_SUFFIX}"
   col=`typeVars.awk -v h=$GIS_FLAG_H -F, $i`

   # Get the raster type
   if `r.info -t $GIS_OPT_RAST | grep -q "FCELL"`; then
 rtype="double precision"
   else
 rtype="int"
   fi
  
   # Grass-specific calls
   v.in.ascii --overwrite input="$i" output="$map" fs="$GIS_OPT_FS" \
 skip=$skip column="$col" x="$GIS_OPT_X" y="$GIS_OPT_Y"
   v.db.addcol map="$map" \
 columns="$GIS_OPT_RAST $rtype"
   v.what.rast vector="$map" column="$GIS_OPT_RAST" \
 raster="$GIS_OPT_RAST"
   v.out.ascii input="$map" fs="," \
 columns="$GIS_OPT_RAST" | \
   awk -F, -v meters="$GIS_FLAG_M" -v head="$GIS_FLAG_H" \
 -v rast="$GIS_OPT_RAST" '
   BEGIN{ if(head == 1) print rast;}
   {
        # Convert Ft. elevation to meters if necessary
        if($4!="" && meters==1) $4=$4 * 0.30480060
        print $4
   }' >"$tmp"
   # Join raster values to input and write to output
   paste -d$GIS_OPT_FS "$i" "$tmp" >"$of"

   # Cleanup
   rm "$tmp"
   if [ $GIS_FLAG_R -eq 1 ] ; then
 g.remove -f --quiet vect=$map
   fi
done
To see the useage for the full script, you can simply start GRASS and from the command line type v.ascii.addrastcol --help. Here's a short example for a random set of 25 points on Ft. Bragg North Carolina.
GRASS 6.4.0RC3 (ftBragg):~ > head locs.csv
"","V1","V2","V3"
"1",594927.510605283,153893.107849400,1
"2",590884.780751329,155855.275043186,2
"3",594095.580107841,155284.234116311,3
"4",605646.865006463,154397.274762958,4
"5",603002.815879479,153284.801461238,5
"6",603453.085431226,152633.248500899,6
"7",602981.046299442,152156.288326646,7
"8",604160.50828444,151921.617483228,8
"9",603752.189821169,152062.642227756,9
GRASS 6.4.0RC3 (ftBragg):~ > v.ascii.addrastcol --quiet -hr \
input=locs.csv rast=iLandcover x=2 y=3 --quiet -r
GRASS 6.4.0RC3 (ftBragg):~ > head locs_isect.csv
"","V1","V2","V3",iLandcover
"1",594927.510605283,153893.107849400,1,3
"2",590884.780751329,155855.275043186,2,3
"3",594095.580107841,155284.234116311,3,1
"4",605646.865006463,154397.274762958,4,3
"5",603002.815879479,153284.801461238,5,3
"6",603453.085431226,152633.248500899,6,3
"7",602981.046299442,152156.288326646,7,3
"8",604160.50828444,151921.617483228,8,1
"9",603752.189821169,152062.642227756,9,3

Labels: , , ,

232 Comments:

«Oldest ‹Older 201 – 232 of 232
Blogger sri harsha said...



TS DSC 2017
Telangana DSC 2017
TS DSC Notification 2017
TS DSC Model Papers 2017
Sakshi TS DSC Model Papers 2017
Namasthe Telangana TS DSC Model Papers 2017
TS DSC District wise Vacancies list 2017

June 4, 2017 at 11:13 AM  
Blogger sunitha vishnu said...


It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
Android Training in Chennai
Ios Training in Chennai

June 8, 2017 at 3:04 AM  
Blogger Karthi Keyan said...

very very amazing explaintion....many things gather about yourself...yes realy i enjoy it
Digital Marketing company in Chennai

June 23, 2017 at 2:44 AM  
Blogger Karthi Keyan said...

very very amazing explaintion....many things gather about yourself...yes realy i enjoy it
Digital Marketing company in Chennai

June 23, 2017 at 3:39 AM  
Blogger sri harsha said...


All the board students can Download Subject wise new Syllabus for all state and central board schools with model papers 2018 for all board of secondary education boards and students can apply scholarship 2018 for ekalyan epass to online application and status check

July 3, 2017 at 11:39 AM  
Blogger Divya madhuri said...

You have provided a nice article, Thank you very much for this one. And I hope this will be useful for many people. And I am waiting for your next post keep on updating these kinds of knowledgeable things
CCNA Openings in Hyderabad .

July 6, 2017 at 8:27 AM  
Blogger kingrani said...

SSC JHT Paper II Exam Results
UPTET Recruitment Notification

August 8, 2017 at 6:34 AM  
Blogger Karthi Keyan said...

Interesting blog post.This blog shows that you have a great future as a content writer.waiting for more updates...

Digital Marketing Company in India

Digital Marketing Services in India

Digital Marketing Agency in India

Digital Marketing Companies in India

August 21, 2017 at 5:35 AM  
Blogger viji said...

you are posting a good information for people and keep maintin it and give more updates too.

Digital Marketing Company in Chennai

Digital Marketing Agency in Chennai

Digital Marketing Companies Chennai

Digital Marketing Services in Chennai

Digital Marketing Company Chennai

Digital Marketing Agency Chennai

August 24, 2017 at 6:12 AM  
Blogger Dharshini Priya said...

I’ve been browsing on-line greater than three hours today, but I never discovered any attention-grabbing article like yours. It is beautiful worth sufficient for me. Personally, if all webmasters and bloggers made good content material as you did, the net will be a lot more helpful than ever before.
Digital Marketing Company in Chennai
SEO Company in Chennai

November 1, 2017 at 8:08 AM  
Blogger Top beauty parlours in Bangalore said...

It's Really A Great Post
Best Elegant IT Services

November 14, 2017 at 11:21 PM  
Blogger Aryan Arps said...


Nice Information. Thanks for sharing such a nice information.
mock test
home remedies
technology
---------------------------------------------------------------------------------
Nice Information. Thanks for sharing such a nice information.
recruitment
jee main
law
gate
school
neet

November 16, 2017 at 4:17 AM  
Blogger sunitha vishnu said...

Really it was an awesome article...very interesting to read..You have provided an nice article....Thanks for sharing..
Android Training in Chennai
Ios Training in Chennai

November 18, 2017 at 2:29 AM  
Blogger Chaithra L said...

It's A Nice Post. Looking for some more stuff.
Best Oracle Training in Bangalore

December 5, 2017 at 2:10 AM  
Blogger Ashwin Ceejo said...


very useful final year projects in chennai, me projects in chennai, btech projects in chennai
mtech projects in chennai, mtech projects in chennai | btech projects in chennai | me projects in chennai | me projects in chennai

February 1, 2018 at 2:00 AM  
Blogger Ancy merina said...

Very amazing blog on a fresh new topic. I am definitely going to use the information provided in this blog because of its novel and fresh. Thanks for sharing such valuable information with everyone.
website developers in bangalore
Website Design and Development Companies in Bangalore
ECommerce Web Design Company in bangalore
Outsource magento ecommerce services india

February 19, 2018 at 3:09 AM  
Blogger joselin mary said...

I got lot of informations from your blog.Please keep us informed like this.And thanks for sharing!!!
Seo Company in India

February 20, 2018 at 6:46 AM  
Blogger Ancy merina said...

I truly like to reading your post. Thank you so much for taking the time to share such a nice information. I'll definitely add this great post in my article section.
Website Design and Development Companies in Bangalore
ECommerce Web Design Company in bangalore

February 24, 2018 at 6:24 AM  
Blogger svr online Training said...

Really Awesome Blog........
Thanks@Salesforce

February 26, 2018 at 11:45 PM  
Blogger Gokul Ravi said...

useful blog
python interview questions
cognos interview questions
perl interview questions
vlsi interview questions
web api interview questions
msbi interview questions

April 21, 2018 at 3:03 AM  
Blogger Gokul Ravi said...

laravel interview questions
aem interview questions
salesforce interview questions
oops abab interview questions
itil interview questions
informatica interview questions
extjs interview questions

April 21, 2018 at 3:04 AM  
Blogger Gokul Ravi said...

sap bi interview questions
hive interview questions
seo interview questions
as400 interview questions
wordpress interview questions
accounting interview questions basic accounting and financial interview questions

April 21, 2018 at 3:05 AM  
Blogger CIIT Noida said...

Ciitnoida provides Core and java training institute in

noida
. We have a team of experienced Java professionals who help our students learn Java with the help of Live Base Projects. The object-

oriented, java training in noida , class-based build

of Java has made it one of most popular programming languages and the demand of professionals with certification in Advance Java training is at an

all-time high not just in India but foreign countries too.

By helping our students understand the fundamentals and Advance concepts of Java, we prepare them for a successful programming career. With over 13

years of sound experience, we have successfully trained hundreds of students in Noida and have been able to turn ourselves into an institute for best

Java training in Noida.

java training institute in noida
java training in noida

May 7, 2018 at 5:37 AM  
Blogger CIIT Noida said...

Ciitnoida provides Core and java training institute in

noida
. We have a team of experienced Java professionals who help our students learn Java with the help of Live Base Projects. The object-

oriented, java training in noida , class-based build

of Java has made it one of most popular programming languages and the demand of professionals with certification in Advance Java training is at an

all-time high not just in India but foreign countries too.

By helping our students understand the fundamentals and Advance concepts of Java, we prepare them for a successful programming career. With over 13

years of sound experience, we have successfully trained hundreds of students in Noida and have been able to turn ourselves into an institute for best

Java training in Noida.

java training institute in noida
java training in noida

May 7, 2018 at 6:21 AM  
Blogger ciit noida said...

Best Summer Internship In Noida

These technologies prepare individuals for fields like software programming, technical support, graphic design, software testing, business analytics. Embedded Systems, Industrial Automation Training and more. Candidates go through a series of comprehensive practical sessions where they work on live problems and implement solutions on real-time basis.
It has a dedicated placement cell which provides 100% placement assistance to students. The benefits that a student gets out of summer training are innumerable. CIITN,Best Summer training Center for B.Tech/CS/CSE/IT/ BCA/MCA/ B.E /M.tech / B.sc/ M.sc/ Engineering Student has already accomplished itself successfully in the field of Training and Development after setting milestones and bringing smiles to the faces of more than 1 Lakh students. CIITN was incorporated in the year 2002 and over the years CIITN has grown remarkably into the greatest training giant of Northern India.
There is no looking back to technology! Those passionate for it need not worry about the professional prospects it offers. Courses and trainings in the field of Computer Science and Applications open a wide array of choices for individuals. All you need to do it prepare yourself right!

Summer Internship Course In Noida

Summer Internship Training In Noida

May 16, 2018 at 3:05 AM  
Blogger ciit noida said...

Best Summer Internship In Noida

These technologies prepare individuals for fields like software programming, technical support, graphic design, software testing, business analytics. Embedded Systems, Industrial Automation Training and more. Candidates go through a series of comprehensive practical sessions where they work on live problems and implement solutions on real-time basis.
It has a dedicated placement cell which provides 100% placement assistance to students. The benefits that a student gets out of summer training are innumerable. CIITN,Best Summer training Center for B.Tech/CS/CSE/IT/ BCA/MCA/ B.E /M.tech / B.sc/ M.sc/ Engineering Student has already accomplished itself successfully in the field of Training and Development after setting milestones and bringing smiles to the faces of more than 1 Lakh students. CIITN was incorporated in the year 2002 and over the years CIITN has grown remarkably into the greatest training giant of Northern India.
There is no looking back to technology! Those passionate for it need not worry about the professional prospects it offers. Courses and trainings in the field of Computer Science and Applications open a wide array of choices for individuals. All you need to do it prepare yourself right!

Summer Internship Course In Noida

Summer Internship Training In Noida

May 16, 2018 at 3:10 AM  
Blogger rohit pal said...

Webtrackker is one only IT company who will provide you best class training with real time working on marketing from last 4 to 8 Years Experience Employee. We make you like a strong technically sound employee with our best class training.


WEBTRACKKER TECHNOLOGY (P) LTD.
C - 67, sector- 63, Noida, India.
F -1 Sector 3 (Near Sector 16 metro station) Noida, India.
+91 - 8802820025
0120-433-0760



Best SAS Training Institute in delhi

SAS Training in Delhi

SAS Training center in Delhi

Best Sap Training Institute in delhi

Best Sap Training center in delhi


Sap Training in delhi

Best Software Testing Training Institute in delhi

Software Testing Training in delhi

Software Testing Training center in delhi


Best Salesforce Training Institute in delhi


Salesforce Training in delhi

Salesforce Training center in delhi

Best Python Training Institute in delhi

Python Training in delhi

Best Python Training center in delhi

Best Android Training Institute In delhi

Android Training In delhi

best Android Training center In delhi

May 17, 2018 at 6:51 AM  
Blogger rohit pal said...

Webtrackker is one only IT company who will provide you best class training with real time working on marketing from last 4 to 8 Years Experience Employee. We make you like a strong technically sound employee with our best class training.


WEBTRACKKER TECHNOLOGY (P) LTD.
C - 67, sector- 63, Noida, India.
F -1 Sector 3 (Near Sector 16 metro station) Noida, India.
+91 - 8802820025
0120-433-0760



Best SAS Training Institute in delhi

SAS Training in Delhi

SAS Training center in Delhi

Best Sap Training Institute in delhi

Best Sap Training center in delhi


Sap Training in delhi

Best Software Testing Training Institute in delhi

Software Testing Training in delhi

Software Testing Training center in delhi


Best Salesforce Training Institute in delhi


Salesforce Training in delhi

Salesforce Training center in delhi

Best Python Training Institute in delhi

Python Training in delhi

Best Python Training center in delhi

Best Android Training Institute In delhi

Android Training In delhi

best Android Training center In delhi

May 17, 2018 at 6:52 AM  
Blogger Kalai chellam said...

Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.

Restaurant in OMR
Apartments in OMR
Villas in OMR
Resorts in OMR

May 19, 2018 at 1:10 AM  
Blogger Kalai chellam said...

Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.

Restaurant in OMR
Apartments in OMR
Villas in OMR
Resorts in OMR

May 19, 2018 at 1:11 AM  
Blogger Harish said...

Thank you for posting this info.

VMware Training in Chennai | Vmware cloud Training

May 25, 2018 at 8:39 AM  
Blogger Harish said...

Thank you for sharing this information.

Best VMware Training in Chennai | VMware Training in Chennai

May 25, 2018 at 8:53 AM  
«Oldest ‹Older 201 – 232 of 232

Post a Comment

Subscribe to Post Comments [Atom]

<< Home