Concerto v3 Tutorials: Create an adaptive test

This tutorial will show you how to create a simple adaptive test on Concerto v3.7 that looks like this:
Concerto v3 Adaptive test

It is assumed that you’ve already taken the Concerto v3 Tutorials: Create a simple test section and are familiar with the basic functions of Concerto v3.

Obviously, this is a very basic test, but it easily shows the idea behind Concerto. Don't forget to add formatting of your choice to make the test professional/pretty! The tests that you develop can be run as a separate website or embedded in the other websites or applications.

It is best to use Google Chrome to run the Concerto administration panel. Please ensure that you have Concerto v3.0 or higher, preferably the latest version. Go to concerto_installation_path/cms/ and login using your credentials. If you haven't installed Concerto on your server, you can use our demo installation at http://concerto.e-psychometrics.com/demo/cms/ to get a free basic account or login to an existing account hosted on our server.

After logging in to Concerto, switch the interface to the SIMPLE mode using the buttons in the top right corner (you will not need advanced functions as for now!):

Step 1: Create HTML Templates

Introduction template

Create a new template called _introduction_adaptive_ and add some suitable text to describe the test and make participants feel comfortable.

Test item template

Create a new template called _test item_adaptive_ and add the following content:

{{question}} insert that will be filled by concerto
"=" sign
An input text field called "response"
- Ensure you call this text field "response" as that's how we refer to it in R code. You can give it a different name, but you will have to change R code accordingly.
- If you don't know how to add a text field, see the Concerto v3.0 Tutorials: Create a simple test section.
A button called "Submit"
For presentation purposes, add the field {{theta}}.

Feedback template

Create another new template called _feedback_adaptive_ that contains:

A suitable title
A few words of empathic feedback
A field {{theta}}

Step 2: Creating an Item Bank/Table from a CSV File

The next step is to create an item bank for the test. Usually, software like R and LTM package are used to generate item parameters. Here, let’s use a mock-up item bank with only one item parameter (difficulty) which can be generated using the Rasch model.

Download and unzip this CSV file http://tinyurl.com/cefwg2z and explore it using Excel or notepad.

Go to the tables tab and click the ‘+ add new object’ icon to add a new table. Assign a meaningful name (E.g. adaptive_test_table) and save it. Click the import table from CSV file icon to import the table that you have downloaded. Concerto will automatically read the data into table format. All you need to do is alter the column names and the data types (see below).

Step 3: Creating a test and adding sections

Go to the ‘tests’ tab on the upper left side of the page. Create a new test by clicking the ‘+ add new object’ icon below the “list of available objects” (it is likely to be empty!). Assign a relevant name for the test and click ‘OK’.

To edit a test at any time, click on the pencil symbol towards the right of the specific test you want to edit. Be careful not to click the ‘bin’ icon (second symbol from the left) unless you wish to delete your test!

This step involves editing your test overall. Click the ‘tests’ tab on the upper left side of the page and click the pencil icon on your specific test to edit it. To add a new section, click the ‘+’ at the right side of the last section. At this point, the last section is ‘1. start’.

Follow the table below to add new sections to your test, using the editor. Please double-check your test logic sections using the text and screen shots provided to avoid any errors that may prevent your test from running currently! Please pay attention to all the letters, as Concerto is case sensitive. Please remember to save your test often and do NOT refresh the page or (accidentally) press ‘go back’.

SECTION TYPE	DETAILS
Load HTML template	Select the introduction template (or whatever you named it)
Set variable	SET VARIABLE theta by R code: "Score not available yet"
Set variable	SET VARIABLE difficulties from table: adaptive_test (or whatever you named it); COLUMNS difficulty
-

SECTION TYPE	DETAILS
Set variable	SET VARIABLE current_item by R code: 50
Set variable	SET VARIABLE responses by R code: NULL
Set variable	SET VARIABLE itempar by R code: as.matrix(cbind(1,difficulties,0,1))
Set variable	SET VARIABLE catBank by R code: createItemBank(itempar,model="1pl")
-

SECTION TYPE	DETAILS
Set variable	SET VARIABLE items_administered by R code: current_item
Set variable	SET VARIABLE question from table: adaptive_test_table (or whatever you named it); COLUMNS content WHERE id equal current_item
Set variable	SET VARIABLE difficulty from table: adaptive_test_table (or whatever you named it); COLUMNSdifficultyWHEREidequalcurrent_item
-

SECTION TYPE	DETAILS
Set variable	SET VARIABLE correct_answer from table: adaptive_test (or whatever you named it); COLUMNS correct_answer WHERE id equal current_item
Load HTML template	Select the test_item_adaptive template (or whatever you named it).
IF statement	IF is.na(response) equal 1 THEN GO TO: *(select the previous section where you load the items* template)**. This step is to ensure that there are no missing responses or non-numeric type responses.
-

SECTION TYPE	DETAILS
Set variable	SET VARIABLE responses by R code: c(responses, correct_answer == response) _Note: The operation correct_answer==response is to compare the correct_answer with response and give value 1 if the statement is true and 0 if false. In this way, the elements in the vector responses will be digits either 1 or 0, corresponding to right or wrong._
Set variable	SET VARIABLE theta by R code: it<-itempar[items_administered,1:4, drop=F] and in the next line thetaEst(it, responses) (Note: the above two lines of code should be in separate lines; do not enter them continuously as one sentence!)
Set variable	SET VARIABLE current_item by R code: nextItem(catBank, theta=theta, out=as.numeric(items_administered))`[[1]]` Please pay attention to all the digits and letters, as well as the brackets, since even a small typo will stop the test from running properly.
Set variable	SET VARIABLE items_administered by R code: c(items_administered, current_item)
-

SECTION TYPE	DETAILS
Set variable	SET VARIABLE nitems by R code: length(items_administered)
IF statement	IF nitems equal or lesser than 15 THEN GO TO (select the section where you set the variable "question")
Load HTML template	Select feedback template (or whatever else you named it)
-
-	*END*

To run your test

Click the Run Test button at the start of the tests section.

Alternatively, use the following URL: (concerto_installation_path)/?tid=(TEST_ID)

Or if using a free account on our server: http://concerto.e-psychometrics.com/demo/?tid=69

_Substitute the last 2 digits in the URL with the_test idof your specific test.

Note: If the test does not work as expected, you will likely see a page with R return code. The second part down the page R output will provide you with the error message which tells you what goes wrong and in which test section.