Efficient ArcServer Cache Management with a Staging Server

Getting a cache built can sometimes be a challenge but caching an ArcServer map service can be important if you want your web apps to display fast. Depending on the subject of your cache and what scales you cache at you can end up with tens or hundreds of gigabytes of data.

Cache tiles

In my office we cache both vector and imagery data for an entire county. We have 20 aerial mosaic data sets dating back to 1937 and several vector data sets that also cover the entire county. Caching all of this can put a real strain on a server and takes up limited server resources that can cause the server to perform poorly.

I’ve found that the best solution is to cache on a staging server and then transfer the cached tiles to a prepared production server. Here is how you can easily do the same:

  1. Create a service on your staging server.
  2. Set up caching in the Services Properties dialog.
  3. Create a service on your production server that has the same name as your staged service.
  4. Set up caching in the Services Properties dialog. From the “Tiling Scheme” drop down select “An existing cached map / image service” and navigate to the staging service. This will import all of your cache advanced setting that you defined on your staging service like scale levels.
  5. Run the Manage Map Server Cache Tiles tool to create the staged cache.
  6. Now you just want to copy your level folder (L00, L01, L02…) located in \arcgisserver\directories\arcgiscache\[your map service name]\_alllayers to the same location on your production server.

At this point your production server should have a full working cache. If you have a cache service that will need to be updated regularly you could even script the whole process and schedule it to run at night or on the weekend.

 

How to Think Like Sherlock Holmes

Mastermind

I often go to the library to try and find something entertaining, intriguing or just different. Last week I found just the thing. It was a book called Master-Mind: How to think like Sherlock Holmes by Maria Konnikova. Now this is the kind of book I love to find. After all, who wouldn’t want to be able to think like the famous detective of 221B  Baker Street?

The title itself intrigued me enough to take it home right away. It promised to free my mind from being common and show me how to be a logical processing machine. Perhaps I could become as great a thinker as Holmes was portrayed to be in his stories.

I won’t go into a deep book review here. If you want to know the details you will have to read it yourself.  I will tell you, however, that after reading the entire thing, my brain is not yet governed by a Holmes-like system of thinking. I have not yet mastered the skills of observation, imagination or deduction that it takes to become a master thinker. But I am further along that road than before reading Mastermind. Many of the book’s insights have actually already helped me solve problems with my programming using the methods that Holmes used to solve his own puzzles.

The book was a fun read. I would encourage you to buy a copy or go to the library and check it out.

Better Basemap Management for the ESRI Javascript API

If you use ESRI’s Javascript API to build web maps you have probably noticed that your ability to manage base map layers is somewhat limited. You really only have three out-of-the-box alternatives:

  1. Define an ArcGIS.com basemap in the map constructor
  2. Create and fill a Basemap Gallery
  3. Switch between two basemaps with Basemap Toggle

But what if you don’t want to use an ArcGIS.com basemap, you want to use something other than thumbnail images to switch between basemaps or you have more than two basemaps to switch between? In that case, you probably need something a little more flexible.

I recently found myself in one of those situations. I wanted to be able to switch between a vector basemap and a basemap of current imagery as well as give viewers the opportunity to choose imagery as far back as the 1930s. I really didn’t want to show thumbnails for every year of imagery. Instead, it seemed to make more sense to allow selection by year from a drop down menu.

So I went ahead and built my own little version of a basemap switcher to handle different situations. To be fair, I could have used the ESRI Basemap Toggle dijit to handle the two main basemaps but I like having more control over my code than Dojo based dijits allow.

I put together a simple map to demonstrate switching multiple basemaps through various means like buttons, thumbnails and drop downs. The html markup and CSS  styling is pretty basic. We create a header as a toolbar (in this case our switcher is the only tool), a div for the map to be injected and markup for the the jQuery UI dialog box that will hold our switching controls.

Edit: FC Basson’s comment got me thinking about the code samples here being too complicated for what they were actually doing. I went back through and removed the toolbar and all jQuery UI and jQuery code. None of these added anything special to the app and since Dojo is already available to us, it should be used instead

The code needed to run the switcher isn’t too complex either. You just need to create an ESRI Javascript map reference, create variable references to the basemaps you want to use then create a function that does the actual changing of the layers.  The rest of the code is just some jQuery Dojo/JavaScript to interact with the controls.

You will notice in the code above that we have create an empty array called currentBasemap.  This array is the key to making the tool work. We store a reference to the currently loaded basemap here. That way we can access it, remove it’s corresponding basemap and load a reference to whatever new basemap that we want to display.

That’s all there is to it. You can get the full code from GitHub. You can also view a working example of the code.

Let me know if you have any suggestions on how to make it better.

Master Google Maps, Research WWII and Convert Data

As I tooled around the web last week looking for anything useful, interesting or entertaining I came across the following four pages that I thought I would share. Hope you like them.

1. Gizmodo.com – 10 Tricks to Make Yourself a Google Maps Master

Google Maps MasterA great up-to-date look at Google Maps and what it takes to master its ins and outs. The bulk of the article deals with using Maps’ search box. That’s great because the search box is full of great functionality that most users probably don’t have experience with. No matter what your level of skill with Google Maps, there is probably a trick or two to learn here.

2. Worldology.com – World War II: Interactive Map

World War II Map

This is a great little Flash based map that gives you an interactive look at the progression of World War II in the European theater. You can visually see the progression of the war and you can read the popup excerpts as you mouse over the different countries. There is even an option to see current political boundaries overlaid on the map. Cool.

3. freegeographytools.com – Download Page

Free stuff is really neat. Free GIS stuff is even neater. Freegeographytools.com is a nifty little blog that focuses on, you guessed it, free geography tools. The download page provides links to many of these tools like data conversion utilities, KML tools and raster tools. It doesn’t seem like the blog has been updated since 2011. There might be better alternative out there but there is still some good information here.

Five Things I Love About the ESRI Javascript API

Having worked with both the ArcGIS Silverlight and Javascript APIs I have to say Javascript has been much more fun. There is just something about working on the front end of a web map and being able to have it do so many amazing things. Here are my top five reasons I enjoy working with this API.

1. Your maps are device independent. Unlike the Flex and Silverlight APIs, Javascript and HTML5 can be viewed on almost any device. The obvious advantage here is that you can reach more people with your spatial data.  Of course having the ability to code for multiple devices means having to customize your code for different screen sizes which can turn into a lot of work. Check out this post from the Treehouse Blog for a great primer on responsive web design.

2. Plenty of code samples and live examples. Code samples are a great way to get your own map up and running quickly without having to figure everything out first. While the samples are not perfect they are a good way to discover map functionality that you might want to incorporate.

3. Support for a ton of layer and data formats. From GeoRSS and GeoJSON to KML and shapefile, this API pretty much covers it.

4. Well written API reference documentation. If you are going to use an API (any API) you want to be able to understand it so you can use it effectively. Good documentation is critical for this. The Esri JS API is (in my own humble opinion) extremely easy to read and understand. Each class has all the information you need to use it without having to decode any meanings. Furthermore, there are links from each class to samples that use it.

5.  Editing tools for online editing of SDE data. Let’s be honest, online editing through a web browser is not a great idea if you are trying to build accurate data. However, there is definitely a place for it. If you are trying to outline an area inhabited by a herd of elk or mark an area of weed infestation, browser editing is probably fine.  It is great just to be able to have this ability so more diverse mapping applications can be developed.

Wow, this list was a lot harder to write than 5 Things I Hate About the ESRI Javascript API. One reason is that there are other APIs out there that do a lot of what the ESRI API does (yes I am thinking about OpenLayers). Another reason is that as I write I think about things I would like ESRI to do better or differently. Oh well, you can’t have everything. Let me know what your thoughts are on the ESRI Javascript API – good or bad.

5 Things I Hate About the ArcGIS Javascript API

Building a mapping application with ESRI’s Javascript API is a lot of fun. But it can also be a real pain. Here are five things that absolutely drive me crazy when writing an ArcGIS Javascript map.

1. It is based on Dojo. It’s probably just my lack of Javascript programming sophistication but I find Dojo a little hard to work with. Although the API documentation has improved in recent versions, I don’t find it very intuitive. If you are a beginner or new to Javascript, the getting started documents are just plain frustrating. Ben Farrell thinks there are seven stages of learning Dojo including anger and acceptance. I’m not sure if there are really only seven stages but I can attest to the anger and acceptance.

2. It uses multiple versions of Dojo. Here I go again with the Dojo thing. Half of the developer samples on the ArcGIS.com developer’s site seem to be written pre version 1.7 while the others are post 1.7 and rely on the AMD module format.  I understand that version transitions of supporting libraries can be difficult but this is ridiculous. Since the ArcGIS API updated to version 3.6, many of the developer samples have been updated to show the AMD require format. Unfortunately, the transition has been slow and there is still a lot of mixed version code and a lot of live samples are broken. If you are going to upgrade to a new version of your API, let’s have things ready to go.  

3. No built-in label engine for dynamic, on-screen annotationIt seems obvious to me that the users of a map would want to mark it up with text; especially in conjunction with a draw toolbar. Apparently it isn’t obvious to the ESRI developers. Instead we are left to work with with the labelPoints function which uses the geometry server to create an unseen point and then puts a text symbol on top of it.

4. The label engine problem above leads me to my next gripe. The LabelPoints function doesn’t allow line breaks. That means that if you want to label a polygon with something like, oh say, area and perimeter, you have to have it all on one line or call the labelPoints function twice and use an offset. How about we get the ability to at least use a newline character (\n) and be done with it.

5. No out-of-the-box support for a table of contents. There is a lot of debate about whether an ArcGIS-like table of contents is a good thing in a web map. No matter what side of the debate you fall on, the fact is that people are used to using them and users often want the ability to turn layers on and off. There are a few projects out there that attempt  to solve the problem but each has its own limitation and problems.  It is a little surprising that ESRI doesn’t have a solution for this one yet. Or, maybe it isn’t really that surprising.

I have plenty more gripes with the Javascript API than the ones listed above but I don’t want to get too depressing. Also, I actually enjoy developing with the API (most days) and there are some great things about it. If you want to add to my list put it in the comments.

CSS Shorthand for Clean Web Map Styles

I see a lot of CSS code written inefficiently and sloppily, even on official and professional sites. It’s not that the code is wrong, it just could be much better. There is really no reason to have to write four lines of code to set a div tag’s margins or style a tag’s background. You can simply use CSS shorthand to get it all into one compact line.

The reason many people don’t use shorthand is because they’ve never taken the time to learn it. Instead, they lean on verbose style rules that spell everything out.   Below you will find a few common examples of CSS shorthand code that might help you when writing your CSS code. This is in no way a definitive guide. It is simply some of the more common properties you will encounter. You can also download a brief CSS shorthand cheat-sheet.

Margin and Padding:

There are four ways to define the four margins and padding of a CSS box.

  1. margin: [top] [right] [bottom] [left];

    Using four values defines each side individually. For example, margin: 1px 2px 3px 4px; would give an element such as a <div> box a top margin of 1px, a right margin of 2px, a bottom margin of 3px and a left margin of 4px.

  2. padding: [top] [right/left] [bottom];

    This notation relies on the same concept as above but now we only need three values to define our padding. padding: 3% 2% 3%; would give our html element a top and bottom padding of 3% and a right and left padding of 2%.

  3. margin: [top/bottom] [right/left];

    Here we only need two values to define our margins. margin 0 auto; gives a zero margin to the top and bottom and centers the <div> within its containing element.

  4. padding: [top/bottom/left/right];

    A single value defines all sides of an element equally. padding: 1em; applies a 1em  pad to the entire html element.

 

Background:

The background property can be defined by this notation -

background: [color] [url(image url)] [repeat] [attachment] [position];

The color property can be defined using hexadecimal(hex) notation, rgb or color names. The image property is written with the prefix “url” followed by parentheses containing the url to the image you want to use for your background. Repeat is used to tile the background image. If you use attachment you can control whether the background image scrolls with the page or not. Position tells the background image what position to start at on the page. The following is an example of a background:

background: #ffffff url(‘greatpic.gif’) repeat-x fixed center;

 

Font:

The font property can be demonstrated by-

font: [size] [weight] [style] [family];

Font size can be defined with a number and a unit such as px, % or em or with size value words like small, medium, large, etc. Weight is simply the thickness of the font characters. Style refers to the values  italic, oblique, normal or inherit. The family refers to the font family like Serif, Georgia or Helvetica. Here is an example:

font: 1em normal italic Helvetica;

 

Border:

Border properties are relatively simple-

border: [width] [style] [color];

The border of an html element is easy to style. You just need to designate the width, color and style. Width can be written using pixels or the pre-determined width designations thin, medium and thick. Style refers to values like solid, dotted and dashed. Color is written the same as in the background property and can be a hex number, rgb numbers or a written color.

border: 2px solid blue;

 

There are a lot more shorthand properties for CSS. Even some of the ones I posted above have further properties that could be included. For a more complete look at CSS shorthand methods check out this page from Mozilla.

Jump Start Your Geospatial Career with This Free E-book

It’s a great time to be a GIS professional. There are no shortages of jobs and the field keeps changing and expanding to include amazing technologies and applications. However, with increasing demand often comes increasing competition. Anyone who wants to enter the geospatial field or advance in it needs to pay attention to fast changing industry needs and make sure they are positioned to fill those needs.

The folks over at DirectionsMag.com have put together a collection of articles dealing with topics of interest to all levels of the geospatial community. It is called GIS Jobs, GISP Certification and Geospatial Careers and can be downloaded at

http://www.directionsmag.com/books/gis-jobs-gisp-certification-and-geospatial-careers/347075.

It is a great little PDF e-book covering things from GIS job interviews to issues surrounding certification. Have a look, it’s free.

 

Easily Export ArcGIS Attribute Table to CSV

For many ArcGIS users, exporting an attribute table to a .csv file or excel format file is a common part of their workflow. Unfortunately, exporting to either of these formats has never been a core functionality of ArcGIS. To overcome this you typically have had two choices: write a script using third party libraries or go through multiple steps exporting to a .txt file and converting it to .csv or .xls.

However, there is a little shortcut you can take to quickly export your data to .csv right from the attribute table.

  • From the table, click Export.
  • Under “Output Table” browse to the folder you want to put your .csv file in, save type as .txt and enter a name for your output.
  • If the ‘.txt’ extension is present in the name the tool will output a normal text file of your data. If, however, your name does not include the ‘.txt’ extension then your output will be a .csv file by default.

This was done with Microsoft Excel installed. I don’t know what the outcome would be if you only had something like OpenOffice installed or no spreadsheet software at all. If anyone has any experience with this feel free to let everyone know in the comments.

4 Essential Maps to Have in an Emergency

Essential Maps for Emergencies

September is National Emergency Preparedness month. Every year throughout the United States there are fires, floods, storms, earthquakes, industrial accidents and even explosions. Sometimes these situations are contained to a small locality where they are easily managed. Other times, emergencies cover large geographic areas and impact the lives of thousands of people. Maps, imagery and GIS have become increasingly important in preparation for and management of emergency situations at federal, state and local levels.

But geospatial tools in an emergency are not just for organizations like FEMA and public safety departments. Individuals have access today to numerous map resources and they could be using them as insurance in the event that a natural or man made disaster should strike in their area. Below are four maps that could give you the upper hand in such a situation.

Home Floor Plan

Every home needs an evacuation plan. A simple pencil sketch of your house will do as you really just need to show the walls and where doors are located. If you want to get more detailed you could use an online floor plan creator like Autodesk Homestyler. Homestyler lets you build detailed, multi-level, 3D floor plans of your home’s interior.

Current Printed Street Map

Google Maps are great for finding the shortest distance by road between locations. Along with other mapping applications like Bing Maps and MapQuest, Google Maps are a good first choice for routing yourself or others to safety. However, you always have to remember that in an emergency situation, Internet and/or cell reception could be disrupted. For this reason, among others, having a printed street map of the city you live in and the surrounding area is a good idea.
Street maps can often be purchased through organizations like AAA or in many gas stations located near the entrances to your city. The local chamber of commerce or visitor center often has maps for free or a low price or they can give you further advice on where to go to get a map. It is true that it is getting harder and harder to find good printed paper maps these days but they are still out there.

Safety Map

Maps are great if you know where you want to go. But in an emergency, unless you have a destination in mind of where you can find help or can meet up with loved ones, a map just shows you many ways to nowhere. At Safety-maps.orgyou can take care of the problem.
            Safety Maps is a free online tool that helps you plan for this situation. You can use it to choose a safe meeting place, print a customized map that specifies where it is, and share this map with your loved ones.
 
Safety-maps.org is a great site that is simple to use but serves a great purpose. Making a safety map is extremely easy. You just search for the area you live in (I used a zip code), position a green cross hair on the place designated for meeting and type a message or directions. Then you create your map and print it. You can fold several copies of the map for different people in your family so each will have one no matter where they are or what the situation.

US Road Map

 
Having a major roads map (or a US road atlas) can help you relocate once you are out of your local area if you need to evacuate quickly. A national level map is also handy for determining the location of disasters outside of your area. Rand McNally is one of the best known makers of quality road atlases.
There you go, four maps that could mean the difference in surviving an emergency situation. Are you prepared? Are there any you would add?