As researchers, we should always be looking for ways to improve our efficiency and increase our productivity. Integrating our tools can be effective on both of those fronts (there is a reason why the Leatherman tool is so popular). By integrating our tools, we have to be less concerned about different file types flying around different programs and more concerned with the fun parts of our work.
R and RStudio, in and of themselves, go a long way towards integrating tools. They allow you to conduct analyses, create documents and figures based upon those analyses, and put those documents and figures into many different forms (PDF, web, etc.). They also provide the power and flexibilty to connect with other tools. The REST API offered by Qualtrics is easily accesible through RStudio. With just a little bit of work, we can connect RStudio and Qualtrics. All you need is API access (you have this if you have account through Notre Dame), your username, and your token (you can find that in Account Settings > Qualtrics IDs > API > Token). Before we jump in, it would be a good idea to familiarize yourself with a few different things. We are going to need to make use of GET and POST functions, so you might want to look into those a bit more. It might be a good idea for you to take a look at the Qualtrics Control Panel REST API Documentation page. Although I will be discussing two control panel functions, you will see that there are many more that we could be interested in using. Let’s briefly consider what we might be able to do with the getSurveys call, GET from httr, and xmlParse and xpathSApply from XML.
With this code:
library(XML); library(httr)
url = "https://survey.qualtrics.com//WRAPI/ControlPanel/api.php?Version=2.4&Request=getSurveys&User=netid5%40nd.edu%23nd&Token=YourToken&Format=XML"
surveynames = GET(url)
xmlNames = xmlParse(surveynames)
surveyID = xpathSApply(xmlNames, "//SurveyID", xmlValue)
surveyNames = xpathSApply(xmlNames, "//SurveyName", xmlValue)
# The previous two lines of code could easily be combined into one line, but I
# wanted you to be able to easily see them both.
surveyNamesID = data.frame(ID = xpathSApply(xmlNames, "//SurveyID", xmlValue), Name = xpathSApply(xmlNames, "//SurveyName", xmlValue))We can get this:
## ID Name
## 110 SV_eg1OWFKL4LDziFD textDataCollection
## 111 SV_egmpMYuQsxnDzvf ggvisExample
## 112 SV_eL2Bb59Y5iO2OCp ManagedRelocationConjoint
## 113 SV_encYVcm3PYO8aix AccountingAlumniSurveyRevisedPCInvite
## 114 SV_eOHCxtAIWn7Jw4l ManagedRelocationCompleteFin
## 115 SV_eUPlXRUbFMdf8od Kroc Professional DevelopmentSo with just a little bit of coding, we have all of our survey names and IDs. This will serve us especially well when we are exporting data from Qualtrics to RStudio. With a little more work, we could get quite a bit more (we could get anything from the XML data).
There are a few other R packages that do some similar things (the exportQualtricsData function that we are going to discuss is very similar to Jason Bryer’s getSurveyResults function in his qualtrics package).
In the previous code chunk, you can see that we need to create a very long and complicated URL with various bits of information (user, token, format). This URL is long, even though that particular request only needs a couple of different fields. Using the power of R, we can create a function that will help to streamline this process.
As previously noted, the following function is very similar to Jason Bryer’s function. However, there are a few special issues for which his function will not account. One such issue is the user name. Our usernames (people affiliated with Notre Dame) include an “@” and a “#nd”. This, in and of itself, does not seem like a big deal. However, you cannot use an @ or a # within an HTML link, so they must be replaced by the appropriate percent encoding (%40 and %23, respectively). Perhaps you are better at remembering things such as these, but I sure would prefer to automate the process as much as possible. I have also included the dropExtra argument. It is a logical argument that will drop the first 10 columns of your Qualtrics output if it is set to TRUE (those are variables for which I sometimes have a use, but not always). I am not sure which REST API version Jason’s function uses, but the one that the following function uses is the most up-to-date.
exportQualtricsData = function (username, token, format, surveyID, dropExtra=FALSE) {
url = paste("https://survey.qualtrics.com//WRAPI/ControlPanel/api.php?Version=2.4&Request=getLegacyResponseData",
"&User=",username,
"&Token=",token,
"&Format=",format,
"&SurveyID=",surveyID,
"&ExportTags=1",
sep="")
url = gsub("[@]","%40",url)
url = gsub("[#]","%23",url)
exportQualtricsData = read.csv(url)
exportQualtricsData = exportQualtricsData[-1, ]
if (dropExtra==TRUE) {
exportQualtricsData[ -c(1:10)]
}
else {exportQualtricsData}
}You only need to enter your username, token, requested format, and the survey id. You can optionally include the dropExtra argument to cut the first 10 columns (it defaults to FALSE if you do not include it).
myQData = exportQualtricsData("yournetid@nd.edu#nd", "YourToken", "CSV", "SurveyIDNumber", dropExtra=TRUE)If you want to further shorten the amount of typing, you could replace this:
url = gsub("[@]","%40",url)
url = gsub("[#]","%23",url)With this:
username = paste0(username, "%40nd.edu%23nd")This would require that you only type everything preceeding the @ (i.e., just your netid). By exporting your data directly from Qualtrics to R, you are saving an enormous amount of time and starting down the path of being able to automate your data downloads.
Qualtrics can do a great many things. However, it is far from efficient (or wise) to rely on Qualtrics for all of your survey creation needs. In general, it is a good idea to start out with a plain text (.txt) file and only move it to Qualtrics once you get everything completely ready. As an aside, I urge people to start with an unformatted text file; there is no reason to create a beautifully-formatted Word document if you are only going to create a web-based survey from it. I flop between Notepad ++ and RStudio for all of my survey editing needs (I use Notepad ++ when editing .QSF files, which is a conversation for another day, or when I need to be able to autocomplete words).
Perhaps you already create your surveys in text files, but remember our previous discussion about tool integration (we do need to be mindful about conceptual continuity)? RStudio allows you to open a new text file right in the editor! We can then use the Qualtrics Advanced Formatting to create a “ready-to-import” survey right in your RStudio.
The Advanced Format is simply a way of specifying parts of questions.
[[AdvancedFormat]]
[[Question:MC:Dropdown]]
This is an example question.
[[Choices]]
Response option 1
Response option 2
Response option 3
[[Question:MC:Select]]
This is another question.
[[Choices]]
Response option 1
Response option 2
Response option 3
[[Question:MC:MultiSelect]]
One last question.
[[Choices]]
Response option 1
Response option 2
Response option 3
[[Question:Matrix]]
This is a matrix question.
[[Choices]]
Pizza
Taco
Burrito
[[AdvancedAnswers]]
[[Answer]]
Response option 1
[[Answer]]
Response option 2
[[Answer]]
Response option 3
It is worth noting that there is also a Simple Format, but it does not have as many features as the Advanced Format. You can go with either one, but your file has to be formatted in one of them for this to work. If you choose not to format your survey with either of the formatting options and still use a text file, you are already saving yourself time.
To push something up to the Qualtrics server, we are going to use a POST request to pass along a link and a file. Like every request we have made so far, this involves creating a very long link. This is too difficult for me to have to do more than once or twice:
httr::POST("https://survey.qualtrics.com//WRAPI/ControlPanel/api.php?Version=2.4&Request=importSurvey&User=netid9%40nd.edu%23nd&Token=YourToken&Format=XML&Name=uploadTest&ImportFormat=TXT", body=list(file=upload_file("C:/Users/netid/Documents/uploadTest.txt")), encode="multipart")If you have not worked with POST commands, they can be somewhat confusing at first (especially if you try to rely on the Qualtrics documentation). Sifting through the link to find the correct places to put the appropriate values is just too much work and it is error prone. It hurts my eyes in ways that I cannot understand.
Instead of enduring a constant struggle, I created a function that will make life a bit easier for you and me. In looking at the following code, you will see that many of the same arguments are used from the exportQualtricsData function.
importQualtricsSurvey = function (username, token, format, surveyName, inputFormat, fileLocation) {
url = paste("https://survey.qualtrics.com//WRAPI/ControlPanel/api.php?Version=2.4&Request=importSurvey",
"&User=",username,
"&Token=",token,
"&Format=",format,
"&Name=",surveyName,
"&ImportFormat=",inputFormat,
sep="")
url = gsub("[@]","%40",url)
url = gsub("[#]", "%23", url)
require(httr)
POST(url, encode="multipart", body=upload_file(paste(fileLocation)))
}After running the code for the function, all you need to do is this:
importQualtricsSurvey("netid@nd.edu#nd", "YourToken", "XML", "surveyName","TXT","survey/file/location/filename.txt")This will create a brand new survey in your Qualtrics account. You will still need to do some work on it (e.g., display logic, CSS customization), but it is there and you did not have to copy and paste something from one program to another! Just think of all the time you saved and the amount of errors you reduced (using regular expressions in RStudio to trim trailing white space should make it worth your time alone).
I understand that this likely represents a significant departure from your normal survey creation routine and I will happily admit that this might take some time to grow on you. To help illustrate my point, let me offer you an example. Imagine, if you will, that you need to create a question that asks people in which state they reside. How would you do that? Let’s say that you probably have the state names somewhere, right? What if one of your colleagues demands to use state abbreviations instead? Do you have those laying around somewhere handy? R has built-in functions for both state abbreviations (state.abb) and state names (state.name).
#For getting state abbreviations or names (just replace state.abb with state.name)
head(as.data.frame(state.abb))## state.abb
## 1 AL
## 2 AK
## 3 AZ
## 4 AR
## 5 CA
## 6 CONow that you have this, you can easily paste this into your text file and use the following code to strip off the numbers and white space.
[1-9][ \t]+|[0-9][0-9][ \t]+In less than 1 minute, you have all of the state names or abbreviations in the survey.
You can also do something similar with countries:
#For getting country names
library(rworldmap)
data(countryRegions)
euroCountries = countryRegions[ which(countryRegions$REGION=="Europe"),]
# You can just replace Europe with any other continent## euroCountries$ADMIN
## 1 Akrotiri Sovereign Base Area
## 2 Aland
## 3 Albania
## 4 Andorra
## 5 Armenia
## 6 AustriaRStudio also has very nice code completion functionality that will be very helpful to us. It will automatically close brackets, parentheses, and quotations. Given that the Advanced Format requires the use of double brackets ([[]]), having them automatically close will be very helpful.
These are but a few brief examples of how using RStudio for survey creation can save you time. It does take some practice, but you will quickly master it with the proper attention. Like with all things that are initially unpleasant, you will be well-served if you force yourself to do it.
Integrating RStudio and Qualtrics will pay huge dividends. It will make you more efficient and will greatly streamline your work flow. Feel free to use the functions that I have offered in this document. By looking at the functions, you should be able to get a pretty good feel for how the functions work and you will be able to easily create your own functions within RStudio to call into the Qualtrics REST API.