Monthly Archives: June 2017

HELP! I just ran a 6 dimensional tabulation with “Gateway To MAST” and my boss told me to prove that the numbers are correct!!

Ok, you can’t find any other site that can duplicate or support the numbers that you just generated. Don’t panic. There’s actually a pretty easy way to verify the numbers.

Pick two of the dimensions. Go to DataFerrett (dataferrett.census.gov). This is the Census Bureau’s online custom tabulation tool. If you don’t know how to use it, go to youtube. There are several tutorials on it. Take the two dimensions that you picked, and use them to create a custom tabulation in Data Ferrett. Make sure you are using the 2014 ACS PUMS data.

Now take the 6 dimensional tabulation that you got from Gateway To MAST, and use the ‘pivot tables’ feature of your spreadsheet program to collapse out all but two of the dimensions. This will add the volumes together across the dimensions (except for person level household counts, LoVal and HiVal – as a general rule, they cannot be added together).

The numbers from the collapsed tabulation should be identical to the numbers from DataFerrett. You’ve just verified two of the dimensions.

Repeat for two other dimensions, and verify those, and so on.

Android App Released Today: Gateway To MAST

Today is the full release of “Gateway To MAST” on the Google Play Store. Gateway is an interface to MAST that allows anyone with an android device to define a custom tabulation. Their query is then sent to a MAST server which creates the tabulation, which is subsequently emailed to the user. Gateway has been in beta test mode in the Google Play Store for about 1.5 weeks, and the beta test has gone very well, so today I moved it from beta test to full release. In beta test, users would typically get their tabulation in about two minutes. The first tabulation that a user does requires email verification, so that first tabulation requires an extra step.

The interface that I used to have here – where you could drop data items into a shopping cart and get a tabulation – was a first step in the right direction, but only a step. I needed a better user interface for MAST, one that can grow to allow user defined dimensions and multiple years of census data. Although Gateway is currently in its infancy, it allows the user to choose up to 8 volumes and 8 dimensions at two levels (Household and Person) simultaneously. It also allows choices of geographies. This video provides an introduction to Gateway. This is version 1.02, so while it doesn’t do much in the way of user-defined dimensions yet, I know of no other point-and-click product that allows the kind of insight into data that this early version of Gateway does.

The video provides an introduction in using Gateway to create a pretty simple single-level tabulation, but it didn’t cover banding (which Gateway supports), so I’ll describe that here. Very often there will be a continuous variable (such as age or household income) that you want to use as a dimension. But only discrete variables can be used as dimensions, so you need to convert the continuous variable into a discrete variable. For example, you might want age bands of 0-5, 6-10, 11+. In that case, when you are choosing dimensions, and you see the “MUST BAND” button, click it, and it will take you to the band editor. In the band editor, click the flashing cursor to make the keyboard appear. Then enter the lowest number of your lowest band (for age, it would probably be zero, for some incomes, it can be negative). Then hit the ‘Done’ button on the keyboard. Next enter the lowest number of the next band. Gateway will calculate the upper levels of the bands, so you only enter the lower levels. After you enter your final lowest level, hit the ‘Done’ button, then (and only then) hit the “BANDING COMPLETE” button. If you want to review the bands that you have created, go to “Review And Submit Your Query” by either swiping from left to right, or clicking the 3-line ‘hamburger’ in the upper left hand corner. That’s all there is to banding. If you need to redo your banding, just go back to the dimension chooser and click the MUST BAND button again.

Creating 8 dimensional tabulations where each dimension can have potentially hundreds of values (or thousands in the case of PUMAs) can create some size issues. There is some internal processing in my system that is very slow, and because of that I’m only allowing tabulations that are 300,000 bytes or less go through the automated processing (bigger tabulations get created, they just don’t get delivered automatically). If you create a tabulation that is greater than 300,000 bytes, you will get an email that tells you how big it was, and suggests that you use the RESTRICT button (in the dimension choosers) to reduce the size of your tabulation without losing any dimensions. For example, there are 480 different occupations in the Occp data item. If you are only interested in one of them, but you choose Occp without restricting it to the one you wanted, your tabulation will be 240X the size that it really needs to be. If you restrict it to the one you want, you will get two categories for Occp: the one you want and “other”. If you have good reason to want a tabulation that is larger than 300,000 bytes, let me know and I’ll try to bypass the automatic processing and deliver it to you manually.