Edit value from the facet

 

I just can't believe we have not yet published a tutorial for this essential OpenRefine feature! 

When you select the edit option from the facet menu, you can edit all matching values in a column, similar to a search and replace function. Of course, the update is recorded in OpenRefine history and can be replicated on a similar dataset.

You can use the edit using facet to quickly clean up typos or misspelled cells. It is also useful to address duplicates that are either not detected by the clustering feature or caught in larger clusters with irrelevant matches. 

In this video example below, we replace the value US  to United States for 45 cells in one shot.

Read More

Using facet to cluster a subset of your data when you have a large number of rows in OpenRefine?

You sometimes when working in OpenRefine have to deal with a large data set that refer to the same entity (person, city, book or any other entries) but using different spelling. When working with a large data set the clustering function can become unresponsive, due to the amount of computing done in order to run the different algorithms.


In this tutorial we will see how you can create subset of records to cluster using facets to better manage the compute load on your machine. If you want to cluster your full data set, you just need to run the cluster function on the different subset created with your facet.

Read More

Increase Set Facet Choice Count

The default maximum number of values displayed in a facet is 2,000. This limitation has been set to prevent Refine to slow down when working with large datasets.

You can override this parameter by changing this value. Click on Set Choice Count Limit and define the new maximum (Refine prompt automatically a value matching the maximum number of value is your current facet).

Read More

Mining and OpenRefine(ing) JISCMail: (from around the web)

A look at OER-DISCUSS [Listserv] JISC CETIS MASHe: a complete tutorial to scrap data from a mailing list and analyse participant and contribution.

Read the full article.

Read More

Finding (Nearly) Duplicate Items in a Data Column (from around the web)

An other great article by Tony Hirst. This tutorial will show you how to use clustering function (ngram and fingerprint) directly in your facet. Really handy.

Read the full article.

Read More

Chit Chat with New Datasets – Facets in OpenRefine (Was /Google Refine/) (from around the web)

A good review of faceting capability including text, numeric, timeline customized and scatterplot facet.

Read the full article 

Read More

Use Google Refine to clean your data for Fulcrum (from around the web)

Use Google Refine to clean your data for Fulcrum

Fulcrum allow to create location-based data collection apps and deploy them to your mobile device. This tutorial show how to use google refine to take advantages of the data you have collected using fulcrum

via Delicious http://docs.fulcrumapp.com/guides/cleaning-up-data-with-google-refine/

Read More

Google Refine Workshop (from around the web)

This tutorial / exercise will walk you through all google refine main functionality. Through it's exercise so you can get your hand on quickly!

Read More

Data exploration tutorial with google refine

Recently, Hugh Stimson published a great article: Data Mining My Old Radio Playlists. His post mix tutorials on php scripting, data cleaning with google refine and data analysis with PostgreSQL.

This answer post demonstrate that data analysis is fully doable in google refine using really basic function (I'll be using GREL function only once for the long tail analysis). I guess also this post is a good illustration of my previous post on data exploration using google refine.

Read More

facet by facet count

Google refine offers the possibilities to facet by name or choice count. This can be useful to focus an analysis or transformation only on value having more than twenty records for example.

Read More

Sort facet by name using toTitlecase(value) expression

When using the text facet option, google refine, sort all available value either by choice count or by name. When sorting by name, all values will be sorted by number first and then by alphabetical order (capital first and then lower case) into something like this:

Read More

Starts or ends with a number

This is a quick and dirty tips to facet cells starting or ending with a number. Regex will be much cleaner for this but unfortunately the grel expression startsWith and endsWith does not support regex :-(

Read More

Extract number from a string

To extract a string of number with a particular length (for example a string of 3 numbers) from a cells we will use the expression match and regex language. This is an easy four steps process

Read More

Extract from twitter hastag and reference

This case has been brought to me by cosmin who wanted to extract hastag from tweets for some analysis and data visualization. Data have been gather using ScraperWiki and their ability to scrap twitter data into one single document (see the video tutorial).

Read More

Video tutorial to clean up your dataset (by free your metadata)

A great video tutorial from free your metadata which show you how to:

Read More

Use google refine to navigate data solely (facet, filter, flag)

Google refine can also be used to solely navigate and explore data without editing them. I find it very useful to explore large data-set thanks to its good user interface, this avoid to develop a specific one to visualize data. In this post I will present the three mains option to interact with the data: facet, text filer and flags.

Read More

Google Refine 2.0 Training video

In this video you will learn to:

Read More

Merge records spread on multiple rows

This article will go through steps to gather on the content of the same row spread on two rows. This method can be extended when a single record is hosted on multiple rows.

Read More

merge 2 columns that have both blank cells

New - March 2020 Update:

• OpenRefine 3.0, we have the coalesce() function:  which natively handles the null correctly. 

• OpenRefine 3.3 introduced a user interface that offers tons of flexibility. See tutorial

The basic function to merge or concatenate data from two different column is the following:

cells["col1"].value + " " + cells["col2"].value

Read More

Add a space to postal code (splitByLength and Merge function)

This short tips explains how to convert postal code store on 6 characters to 7 by adding a space after 3 digits. We will use splitByLength (see related video) and merge multiple column into one functions.

Read More