Mapping your web server visitors in Google Earth, Part 3 (conclusion)
A few days later, but here’s the conclusion to the story. In part 2, I showed how to parse out the apache log file to get a list of IP addresses. Now all you have to do is, one by one, geolocate those addresses and write them out to a KML doc that gets displayed in Google Earth (or even Google Maps).
For geolocating, I’m using the free product by MaxMind (http://www.maxmind.com/app/ip-location) to do the lookups for the geolocation. Here’s the code to do the geolocation and write out to KML.
import com.maxmind.geoip.*;
import java.io.*;
/* sample of how to use the GeoIP Java API with GeoIP City database */
/* Usage: java CityLookupTest 64.4.4.4 */
class CityLookup {
public static void main(String[] args) {
try {
LookupService cl = new LookupService(args[0],
LookupService.GEOIP_STANDARD);
BufferedReader in = new BufferedReader(new FileReader(args[1]));
BufferedWriter out = new BufferedWriter(new FileWriter(args[2]));
String str;
out.write("n");
out.write("n");
out.write("n");
while ((str = in.readLine()) != null) {
Location loc = cl.getLocation(str);
if (loc == null) {
System.out.println("loc null");
} else {
out.write("
");
out.write("" + str + " ");
out.write("" + str + " ");
out.write("
");
out.write("" + loc.latitude + ","
+ loc.longitude + " ");
out.write(" ");
out.write(" n");
}
}
in.close();
out.write(" n");
out.write(" n");
out.close();
cl.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Now, all you have to do is import the file into Google Earth, or if you have a publicly available web server, you can put the file there, and have google maps pick it up by passing the file as a parameter.
I hope you’ve enjoyed reading this 3-part tutorial as much as I’ve enjoyed writing it. This could be the start of a fun project. Hope you enjoy!
Mapping your web server visitors in Google Earth, Part 2
Part 1 of this series talked at a high level about what was to be accomplished – viewing your web server visitors geographically on a map. It’s got a cool factor, but actually is quite useful for some folks.
This part of the series will show you how to take the Apache web server’s combined log format and get out the ip addresses that have visited your site. This code just parses a file, and takes the first token (first text before a space) and writes out a new line for each address. Alternatively, you could, instead of printing each line out, you could add them to a StringBuffer for better performance, then print them out. A reasonable idea would be to add them to a concrete implementation of java.util.Set so you only saw unique IP addresses. Anyhow, here’s the starter code.
import java.io.*;
public class ParseLogFile {
public static void main(String[] args) {
String newline = System.getProperty("line.separator");
try {
BufferedReader in = new BufferedReader(new FileReader(args[0]));
BufferedWriter out = new BufferedWriter(new FileWriter(args[1]));
String str;
while ((str = in.readLine()) != null) {
String[] strs = str.split(" ");
out.write(strs[0] + newline);
}
in.close();
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
This should read in the apache log file and write out to a file with a long list of ip addresses / domain names. The above code is very simple and fairly unnecessary. You could easily use OS utilities to accomplish the same thing, and you could do this for any type of log file, really. This was just an example.
The last step is taking the output file, and then geolocating each address, and creating some KML out of the addresses. That’s gonna be saved for part 3. See you soon.
Mapping your web server visitors in Google Earth, Part 1
OK, so this is mostly just for the cool factor. However, it is somewhat useful to see where your visitors are coming from at a glance. This type of project would be a nice project for a display wall in a server room somewhere. This example (explained fully other parts to come over the next few days) is simple in that it only looks at 1 apache log file format. It uses some open source information for doing it’s mapping, and then generates the KML for Google Earth to display a bunch of little dots on the earth for every access on your web server. The extensions to this project could be endless, starting with supporting many more file formats, and creating a prettier version of this – mine’s fairly rough. Well, more to come, but you’ll see the code over the next few days.