1.

Can I enter data directly into jMetrik?
No. jMetrik only provides a view of the data. If you must enter data manually, use a program like OpenOffice Calc, http://www.openoffice.org/ (it's a free download). Save the file as a CSV (comma delimited) file. Import the CSV file into jMetrik. The first row of data in the CSV file may contain variable names, but variable names are not required. You should have one row of data for each examinee and one column for each variable.

2.

What is the maximum sample size in jMetrik?
There is no sample size limit in jMetrik. We have tested jMetrik with data files as large as 1,000,000 examinees using the default memory allocation of 256MB. If you have data files larger than 1,000,000 examinees, you may want to increase memory allocation to improve performance. See the Tips and Tricks page for instructions on increasing the memory allocation.

3.

What is the maximum number of variables allowed in jMetrik?
The maximum number of variables in each table is 1,012. However, you can have an unlimited number of tables. If you have a data file with more than 1,012 variables, consider dividing it into multiple files and import them as multiple tables.

4.

Do you have a user manual?
No. We are developing some basic support materials that users can follow to get started. These materials will be available for free on this website. (See the Help menu above.) If you have a question, please send an email to support@itemanalysis.com.

5.

How can I learn more about jMetrik and its capabilities?
We are now providing online workshops and videos. These workshops cover the theory of all the methods included in jMetrik and an explanation of all of the options found in jMetrik and a description of the output. To maximize the utility of jMetrik, please register for one of our workshops. Workshops are held in January and August of each year. All you need to participate in an internet connection. For more information, see http://curry.virginia.edu/community-programs/conferences/jMetrik.

6.

I get an error message when I try to import my data. What is wrong?
You most likely have a problem with your data file. Open your file in a plain text editor and check for extra text beyond the last column or after the last examinee. If you created your data file with Microsoft Excel or some other spreadsheet program, there is a good chance that you unknowingly created extra information in the file. For example, if you remove an examinee from the data file in Microsoft Excel by using the "Clear" option in the context menu, then the examinee remains in the data file but does not have any data. This results creates an empty row of data. To properly remove data from Microsoft Excel, select the row (or column) and click "Delete" from the context menu. That will elimite the data corretly.

Another possible problem is that you do not have unique variable names in your data file. jMetrik requires unique variable names that are no longer than 10 characters. Variable names longer than 10 characters will be truncated possibly resulting in non-unique variable names.

7.

Can I save a graph produced by jMetrik?
You can save all graphs created in jMetrik. After creating a graph, right click (or CTRL click on a Mac) the graph to display a context menu. One of the options is for saving the graph as a PNG file. (Other options allow you to customize various features of the graph.)

8.

What is the information in the DIF analysis output?
There are several columns in the DIF analysis output. The first column "Item" is the name of the item. The second and third columns are the Mantel-Haesnzel chi-square and associated p-value. Note that jMetrik uses the Cochran-Mantel-Haenszel for stratified 2 x k tables. It is a generalization of the Mantel-Haenszel for stratified 2 x 2 tables and it works with binary and polytomous items. One difference between the Cochran-Mantel-Haesnzel for binary items and the Mantel-Haenszel is that the former does not use a correction for continuity. Therefore, with binary items the Mantel-Haenszel value reported by jMetrik may slightly differ from the Mantel-Haenszel reported by other programs.

The "Valid N" column lists the number of examinees involved in the Mantel-Haenszel statistic. Examinees that failed to provide a value for the DIF group code are deleted from the analysis. Also, any stratum table with only one examinee is not included in the analysis. As such, you could have several examinees eliminated from the analysis if you have several tables with only a single examinee. In this situation, try using the "Deciles" or "quntiles" options to preserve more data.

The "E.S. (95% C. I.)" columns list the effect size (E.S.) and 95% confidence interval for the effect size, respectively. For binary items, the effect size is the common odds ratio (COR). You can optionally convert this value to the ETS Delta metric. For polytomous items, the effect size is the standardized P-DIF statistic (sP-DIF).

The final column ("Class") is the ETS DIF classification level. The possible classifications for binary items are A, B, and C, while the possible classification levels for polytomous items are AA, BB, and CC. A and AA items show little to no DIF. B and BB suggest moderate amounts of DIF. C and CC items suggest a large amount of DIF. These classifications are a function of statistical and practical significance. The rules for binary items are:

• A item: (a) Chi-square p-value > 0.05 or (b) the COR is strictly between 0.65 and 1.53.
• B item: not and A or C item
• C item: (a) COR < 0.53 AND the upper bound of the 95% confidence interval for the COR is less than 0.65, or (b) COR > 1.89 AND the lower bound for the 95% confidence interval for the COR is greater than 1.53.
  

Given that polytomous items use a different effect size, the classificaiton rules are different. The rules for polytomous items are are based on dividing sP-DIF (the value in the output) by the item score range to limit values to the interval from 0 to 1. Call this new value sP-DIF*. It is not displayed in the output. This change allows the rules developed for binary item P-DIF to be applied to polytomous items. The rules for polytomous items are:

• AA item: sP-DIF* < 0.05
• BB item: 0.05 >= sP-DIF* <0.10
• CC item: sP-DIF* >= 0.10

Each DIF classification also includes a sign. A "+" sign (without the quotes) indicates that the item favors the focal group. A "-" indicates that teh item favors the reference group. More information about the DIf classification levels are available in the following articles.

Dorans, N. J., Schmitt, A. P., & Bleistein, C. A. (1992). The standardization approach to assessing comprehensive differential item functioning. Journal of Educational Measurement, 29, 309-319.

Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation, Applied Psychological Measurement, 19, 23-37.

Zwick, R., & Ercikan, K. (1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55-66.