DTEM 2411 / Fall 2020

Digital Research Methods

Class Time:      Tuesday 2:30-5:15

Professor:        Kara Van Cleaf | kvancleaf@fordham.edu | Office @ Martino Hall (45 Columbus Ave) Room

Office Hours:  Monday 1-3 | Tuesday 12:30 – 2

Links:              CMS Facebook: www.facebook.com/FordhamCMS | Twitter: www.Twitter.com/FordhamCMS

OVERVIEW

Course Description

Digital technologies affect every area of social life, from personal identity, to interaction with others, to broad social and political arenas. Digital technologies have also deeply impacted scholarship and research in the humanities and the social sciences. How can we investigate the impacts of digital technologies accurately? How do academics and industry professionals use social media, “big data,” and the like to answer puzzling questions? This course provides an overview of and hands-on approach to contemporary digital research methods, including ethnography, interviews, focus groups, metrics and analytics, and polling and surveys. Students will become familiar with basic research methods used in both academic and professional contexts. Four-credit courses that meet for 150 minutes per week require three additional hours of class preparation per week on the part of the student in lieu of an additional hour of formal instruction.

WEEKLY SCHEDULE

Provided is a broad overview of the sequence of topics.

Date	Read | Share | Watch | Discuss	To do, this week or soon:
9/6	Introduction	Next Class 9/13: Find an interesting research article on a topic you care about Summarize the article in 500 words or less. Include the following: What is the research question? Is this a quantitative, qualitative, mixed methods research approach? Which worldview (postpostivist, transformative, constuctivist, pragmatist) would you say best characterizes this study? Why? What did you find interesting about the findings? Did you note any major issues with the research design/approach?
9/13	Design Research
	Discuss: Creswell, Research Design – Chapter 1 (pp 30-53) https://drive.google.com/file/d/106zS3MawBzs0mBUTlSWrvx2knq1hemfJ/view   Beyond the coded gaze	Starter pack assignment (due 9/27) In groups of two. Locate an online culture you are semi-familiar with and observe it for a week. Decide on a way to record your observations, which should include: participants, followers, platform, output, hashtags. Also pay attention to the content and the messages communicated. What symbols have meaning? What unique language is used?   Start gathering info to create a “starter pack” for your chosen online group. Create a google doc to hold your field notes, links, and any other data you use to study and understand your group. You will turn your field notes in to me.
9/20	Digital Cultures/Digital Ethnography cont
	Read: Ethnographies of Online Communities   Is summer over…forever? (try this link)   Doing Digital Ethnography (here)   Unpacking the meaning of the starter pack	Starter pack discussions
9/27	Share starter packs
	Focus Groups Read: Focus Groups Can Be Fun	Focus group meetings.
		Due Next Class.  For this assignment you will be designing a focus group to better collect Fordham students’ opinions, ideas, or beliefs on a topic or product/service.  The main constraint for this assignment are the participants – Fordham students. You will focus on Fordham students, so that we can actually execute our focus groups with our classmates and report our data. Besides this constraint, you can focus on any topic or product/service you want to frame your focus group research question.  1. Research Question The first part of your assignment will be to identify a research question. Below are some templates you might use. Notice the way the questions lend themselves to qualitative data and not yes/no answers such as …Do Fordham students want better campus food? How do [Fordham students] feel about [something]? What kinds of suggestions do [Fordham students] have about [something]? What do [Fordham students] believe about [something]? 2. Why this exploration? (max of 300 words) What do you already know about this question (if anything)? What do you expect to find? What about Fordham students in particular (if anything) might make this exploration interesting? 3. What are three activities you can use in your focus group to collect data? Go beyond just a question and answer structure and think of ways you can better get at the kind of responses you feel you could analyze to truly address your research question. Challenge yourself to get creative! …. Even if you don’t end up deciding to use this activity. Use the reading to ground some of your ideas for activities. For each activity idea use the structure below to help guide your thinking. Think about what the participant will do, and what kind of data you will be able to collect from the activity. Again don’t worry about feasibility at this point. Activity X In this activity the participants will be asked to XYZ for about Y minutes. After doing XYZ, we will then move to ABC for Z minutes.  The data generated from this activity will be [something].
10/4	Focus Groups cont
	This week you should run your focus groups. Today in class, your group should do this: Design a 30-45 minute focus group for 4-5 participants.  Your focus group protocol should include:  Names of all the people in your group A sequence of what the participants will do with how long they will engage in that activity.  A link to your mural board Submit a link to your focus group protocol as a shared Google doc. All students in the same group should submit the same document link.	For next week: Set up a Mural account and create a Mural Board for your team.
	Run Focus groups before 10/11
10/11	Focus Group analysis	Affinity mapping
	Affinity mapping:  https://zacknaylor.medium.com/how-to-create-an-affinity-diagram-for-ux-research-cdc08489952d   This week, your team should work on your focus group analysis. You should write up ONE paper and create a short presentation on Canva to share your findings with the class. You will share these next week. Final analysis due: 10/18   Using Canva, generate a summary report of your group’s findings. Create a page for each section: Title: Name of your study; Group members; Study Design: Research question; How many participants did you have; How long was the workshop; What was the protocol for the data collection; Analysis: Snapshot of your affinity map; Description of the key themes Discussion: What are the key takeaways in regards to your initial research question?  Export your Canva project as a PDF and attach to your submission.  Here is a Canva tutorial.	Using the research question that guided your focus group, think about the kinds of things you want to be able to “pick up” from the focus group video and the mural board. If your focus is on understanding why young people are apathetic about voting, then the kinds of things you might want to look out for might be: reasoning on impact voting has on society; experiences with voting; ideas on how to engage young voters; challenges to engaging youth voters, etc.The idea is to “sensitize” your team to certain things you feel will enhance your ability to address your research question. This is a balance of looking for answers to questions you directly asked, but also things you may not have directly asked about, but might have been revealed somehow. These will guide your observations.  Using a shared mural board, your team will create the following spaces and materials: Generate observations on the Mural board artifact, as post-its.  Generate observations on the focus group video, as post-its.  WHAT TO SUBMIT Submit a link to your mural board.
10/18	Focus group Canva presentations
	Surveys! Read: Writing Survey Questions Survey Basics Overview   Watch this Youtube video:  https://www.youtube.com/watch?v=Rf-fIpB4D50	We will create new teams for our survey unit. Today you will get together and brainstorm, as well as sign up for a free account with Qualtrics.   By next week: Use Qualtrics to create a survey. Demographic: No more than 3 multiple choice questions No more than 2 likert scale questions No more than 2 open-ended questions Make sure these are STRONG open-ended questions – they will likely yield a response beyond a few words.  Do not lead the participant too much. Try to be neutral.   No more than 2 additional questions of any question type.  Share your survey with another group for feedback. Revise your survey accordingly. Send revised survey to me to distribute to the class.
10/25	Surveys cont
	Sentiment analysis:   Read: Sentiment Analysis for Small and Big Data   YouTube:  https://www.youtube.com/watch?v=AJVP96tAWxw	Revised surveys sent to the class.   Run surveys, gather data, and code the open ended questions (code with negative, positive, etc).
11/1	Run Surveys	Collect data from surveys and prepare a brief presentation for next week. Presentations should be slides, with graphs. A short write up for me.
11/8	Survey Presentations   Text Mining: An introduction to theory and some applications	Discuss limitations of surveys and sentiment analysis.
	Text Mining, Data Scraping, Oh my! Read   How to analyze twitter data   Watch: What is an API? https://www.youtube.com/watch?v=s7wmiS2mSXY Using a REST API to get social media data https://www.youtube.com/watch?v=7YcW25PHnAA Using MaxQDA to Analyze Twitter Data https://www.maxqda.com/how-to-analyze-twitter-data	Twitter Set up a MaxQDA accountFormulate a research topic/question
11/15	Twitter projects	Move the information from your Google Slide to your doc. Make any revisions based on feedback.  Import all of your Tweets using MaxQDA (if you haven’t already done so) Run any autocodes on authors and hashtags using MaxQDA Twitter Autocode function Create a code called “Sample” Use existing autocodes generated to sift through sample using your inclusion criteria Use keywords and the lexical search to sift through Tweets. Using your inclusion criteria, code any retrieved segments as “Sample” Document the kinds of keywords you used in your analytical search in your Google Doc Set a timer for 30 min, and manually read through as many tweets as you can (rename the documents you’ve read through). As you read, use your inclusion criteria to code segments as “Sample” If you need to adjust your revision criteria or research questions – now is the time.  Manually read through the remaining Tweets, adding to your sample code Add to your Google Doc the following: Create a section called “Methods” Create a sub-section called “Data” Describe the criteria you used to import the initial data corpus Describe your inclusion criteria Tell us how you used analytical searches (what keywords) Tell us how you manually read through the Tweets (how many did you look through) Tell us how many segments you have in your “Sample” code
11/29		Organize your codes into 2-5 broad thematic codes. These should be new codes that you can drag-and-drop other codes into.  Right click on each new thematic code and add a memo that describes the theme. Provide the following in your memo: How would you describe this theme?  What kind of tweets characterize this theme? What is most interesting about this theme? Go to Reports > Export > Project Components as Excel File Submit the excel file for this assignment.
12/6	Thematic reports on twitter data Due 12/
12/	Final

COURSE MATERIALS

• MaxQDA 2018 or more recent software
  • Requires Windows (7/8/10) or Mac (10.9+) computer. More info on requirements:
    https://www.maxqda.com/products/system-requirements#panel04
  • Download and install using the student license provided below. Please do not share with others outside of the course, as there are a limited number of licenses.
  • License info coming soon.

---

COURSE GOALS

• Understand how to formulate strong research questions to guide method selection and parameters
• Engage in data collection, cleaning, and analysis using different research methods
• Understand how to write up findings and results for the public
• Reflect on the challenges and limitations of tools, approaches, and research agendas

HOW TO SUCCEED IN THE COURSE

First, do the readings! Or tell me why they aren’t useful, so I can reconsider their use as part of this course. The readings are there to support the hands-on learning aspect of the course.

There is so much to cover, but the focus was on providing a tight connection between hands-on activities and scholarship/techniques/history around user experience design.

Second, set aside some time or take the course another time. The nature of applied courses means you need to actually engage in the research process. It can be challenging to compress a complex process into a semester course, but we try. The result is that you need to set aside time. Put it in your calendar, when you will sit down and do the readings. Look at the schedule of assignments, and set aside time when you will plan things out, and when you will actually implement the plan.

Third, we are going to be working in groups this semester. We will set aside time each class to do group check-ins, creating work plans, and tracking our progress. But at the end of the day, staying on top of your work and contributing positively to your group is CRITICAL to your success in class, but also will be critical for your career (where you will always have to work in groups at some point).

In short, from my experience I can tell you the three things that hamper success in this course is: (i) not doing the readings and therefore not being able to do the assignments, and (ii) not setting aside enough time between classes to do the assignments (these take time to plan out and conduct), and (iii) not being a reliable contributor to your group project work.

---

GRADING

You should think about this course, not as a traditional class, where you sit and listen to lectures and take tests. Instead, this is a studio class where you are expected to engage and do. The grades you get are not a reflection on some static knowledge you may or may not have, but whether you engage and whether the things that you do show a developing, quality skillset as a designer.

Knowing this, you should also know that sometimes in life we don’t engage fully, and sometimes we do things (especially if we are novices at something) that are not perfect. You’ll also do these things in our class, and your “grades” will reflect our imperfect learning process. So, you should not see your grades as an assessment of you. It’s a feedback mechanism about how you’re engaging and the quality of what you are doing. If you don’t do well one week, ask your professor about what you can do next week to improve. Be ok with making mistakes and getting less than perfect grades. But if you see yourself getting poor assessments for an extended period of time, that’s a signal that you should talk to your professor (me) about what you should change in terms of your process, and how to improve. Keep track of yourself and stay engaged.

Students’ course work will be evaluated as follows:

1. Engagement (10 %) Present, and active participation in class discussions, exercises, and group work.
2. Projects: Starter Pack (15%) / Focus Group (20%) / Surveys (20%) / Twitter Report (20%)
3. Final (15%)

Grade Scale

The final grade for the course is based on the following percentage scale:

• A (94 or above),  A- (90–93)
• B+ (87–89), B (83–86), B- (80–82)
• C+ 77–79, C 73–76, C- 70–72
• D+ 67–69, D 63–66, D- 60–62
• F 59 or below.

---

POLICIES

Attendance

Repeated, unexcused absences from class will affect your final grade.  Missing more than two of our 14 classes will result in a “0” for the “preparation and participation portion of the course grade.  (In documented cases of serious illness or emergency, of course, this penalty will be waived.) Consistent tardiness will be treated as unexcused absences. Make-up work will be allowed without penalty for excused absences only. For unexcused absences, there will be a full letter grade loss on the evaluation of any assignment for each day that it is late. Make-up work will not be accepted one week after the original due date. Consult the Fordham attendance policy for additional details.

Late Submissions

Will not be accepted unless you have pre-arranged an extension at least 48 hours in advance. (Again, in documented cases of serious illness or emergency, of course, this policy will be amended.)

Plagiarism

Not good. Will not be tolerated. It is intellectual theft!  A plagiarized assignment or portion of an assignment will result in failing the course. Period. Students should review the Policy on Academic Integrity stated in the 2008-2009 Student Handbook: 
A University, by its nature, strives to foster and recognize originality of thought. Originality can only be recognized, however, when people acknowledge the sources of ideas or works that are not their own. Therefore, students must maintain the highest standards with regards to honesty, effort and performance. Violations of academic integrity include, but are not limited to: plagiarism, cheating on exams, false authorship, and destruction of library materials needed for a course.  This policy gives definitions and instances of violations of academic integrity, the procedures used to arrive at a judgment, possible sanctions, and the process of appeal. Please refer to your Student Handbook for the Policy on Academic Integrity: 
http://www.fordham.edu/images/student_activities/lc/deanofstudents/graduate%20guide%202009-10.pdf

Students with Disabilities

If you have a condition that may interfere with your  ability to participate in the activities, coursework, or assessment of the object of this course, you are entitled to accommodations. Please schedule an appointment to speak with me immediately or you may go to the Office of Disability Services (Room 402-DMcMahon Hall, x6282). Under the Americans with Disabilities Act and Section 504 of the Vocational Rehabilitation Act of 1973, all students, with or without disabilities, are entitled to equal access to the programs and activities of Fordham University.

Please Check your Fordham Email Every Day

Throughout the term, I may send class-wide emails calling your attention to particular articles or asking you to bring in examples.

---

STUDENT RESOURCES

Writing Center: If you struggle with the writing required in this course, please make use of the Writing Center. For more information about their services see the Writing Center’s website.

Counseling and Psychological Services: We all experience emotional distress and personal difficulties as a normal part of life.  As your instructor, I am not qualified to serve as your counselor. However, Fordham’s office for Counseling and Psychological Services (CPS) provides free and confidential mental health services that are not connected to your academic record in any way. If you are experiencing mental health distress, I strongly encourage you to take advantage of CPS’ services. For more information about CPS, please visit their website at www.fordham.edu/counseling. To make an appointment, please call 718-817-3725 (RH) or 212-636-6225 (LC).

Academic Coaching: Any student who struggles with time-management and/or organization is invited to one-on-one academic coaching sessions (typically once a week for 45 minutes). Please contact disabilityservices@fordham.edu for more information.