Q: The text covers all areas and ideas of the subject appropriately and provides an effective index and/or glossary
1. The textbook overall is out of date in terms of the examples, the newest reference I could find in the book was from 2013 (now 8 years old!), I know from personal experience that students generally prefer new examples of research and want to know what the most cutting-edge research is in the sub-fields they want to go into. This book seems to be lacking in that – the example in the first chapter of Mehl and colleagues (2007) on gender differences in “talkativeness” feels outdated and overly discussed.
2. The APA style chapter is also out of date (not in line with current APA style 7th edition standards) and misses a number of important aspects related to APA style, including in-text citations, and the different headings levels and the examples of "hanging" indents (on page 194) are not indented far enough, the example of a reference section on page looks like it’s 1.5 spaced instead of double spaced.
3. There is very little discussion of naturalistic observational research, for instance there’s no discussion of the difference between the use of natural vs. contrived settings or the different methods for quantifying observations (e.g., interval method, frequency method). Also, I don’t understand why the “Qualitative” research section is not included with the other non-experimental designs – interviews, focus groups and narrative approaches are all non-experimental/descriptive research strategies, yet the qualitative section is lumped in with the quasi-experimental section. There’s also no mention of “case studies” in the qualitative section, or ethnographic research, or community research.
4. The lack of Index for searching specific terms in the book or a true Reference section at the back make the book overly difficult to use. Also, the fact that References are given throughout the book as footnotes goes against APA style which is the style, we expect the students to be writing in – setting a bad example for them.
Comprehensiveness Rating: 2 out of 5
Q: Content is accurate, error-free and unbiased
1. The book on page. 94 discusses “The Four Big Validities” and gives those as “internal” “external” “construct” and “statistical validity” – this oversimplifies the topic of validity. There’s the validity of the conclusions we make as researchers from our results – this is the “internal-external” validity dimension, and then there’s the validity of the outcome of the testing instruments we use, which breaks down into 1. Face validity, 2. Construct validity, 3. Criterion-related validity and 4. Content validity. The textbook does not really talk about construct validity at all (perhaps the authors thought it was too similar to the idea of content validity – but they are very different – construct validity is how well we operationally define the variable we want to investigate: e.g., “measuring happiness by attaching recording electrodes to a person’s face will give us a better measure of their happiness than if I attached them to their arm” content validity is how thoroughly I’m measuring the content area of interest – for instance if I’m measuring happiness maybe I want to get both a measure of the muscle activity in the person’s face AND a subjective rating from them about their emotional state. I would also like to note that “discriminant validity” is a sub-type of criterion-related validity and not its own validity type.
2. The textbook gives some very confusing explanations regarding statistics – some of which presumes more detailed knowledge about a topic than students are likely to have, one that I noted is on page 250 regarding p-values where it states “… a misguided researcher might say that because the p value is .02, there is only a 2% chance that the result is due to chance and a 98% chance that it reflects a real relationship in the population. But this is incorrect. The p value is really the probability of a result at least as extreme as the sample result if the null hypothesis were true” No student I can think of would appreciate that difference, and I would have to spend an entire class clarifying this one paragraph.
3. There are some implicit gender biases in language use throughout. I expect this textbook was written a while ago, so I don't want to be too hard on the authors here - but there are aspects of implicit biases in the writing that generally do not go over well in a class in 2021 - to give an example, on page 192 while discussing avoiding biased language - biased language is actually used, where it says "Instead of... man, men" "Use... men and women, people" - the suggested alteration puts "men" in front of "women" as a general rule for being “gender neutral”. It’s a subtle bias - but still a bias, in other places in the text “women and men” are used as well, but the more modern approach would be to use gender neutral pronouns like "they" or "them", etc. or just use a person’s name directly.
4. I would also note that the images are sometimes inconsistently sized or feel unnecessarily big in relation to the textbook, for example: on page 160, Figure 9.2 gives examples of rating scales for closed-ended questions in a diagram that takes up too much space on the page (and too much dead space around it), some of the picture (e.g., the image on p. 202 feels blown up in size to the point of being blurry).
Content Accuracy Rating: 3 out of 5
Q: Content is up-to-date, but not in a way that will quickly make the text obsolete within a short period of time. The text is written and/or arranged in such a way that necessary updates will be relatively easy and straightforward to implement
1. Most students are going to be analyzing their data by using some kind of statistical software suite, including: Excel, SPSS, JASP or possibly R. It would make more sense to show how to analyze data within the context of one of these programs (e.g., looking at the output generated by those programs), over the current approach which describes how to do it by hand - which is an obsolete method for instructing students on this.
2. As I've already noted - not having a single reference beyond 2013 is fairly jarring for a student taking a course in 2021, for instance the textbook I am currently using right now provides census data from 2018 in a number of discussions, and the TCPS2 ethical statement has since been updated as well (2018), while the latest APA style manual is 2020. I am very hesitant to use this book as it feels anachronistic.
Relevance Rating: 1 out of 5
Q: The text is written in lucid, accessible prose, and provides adequate context for any jargon/technical terminology used
The clarity of the book is overall pretty good - outside of the noted discussion on p-values.
Clarity Rating: 3 out of 5
Q: The text is internally consistent in terms of terminology and framework
One thing that struck me was the lack of framework, there is no overview chapter or preface that explains the "style" or "aim" of the book (e.g., is the book aiming to improve critical reasoning skills? Is it aimed at engaging student interest in the topic?) , there could be a lot more direction in terms of how students should use the book. For instance there are "Exercises" at the end of each chapter - are those intended to facilitate the learning of the objectives? If the students do those exercises will they meet the learning objectives for that chapter? I'm a bit skeptical that they would - but there needs to be more direction in terms of how the students should approach that. Simpler "Learning Checks" might be more satisfactory in accomplishing that. The textbook is very minimalistic in its approach to engaging readers with exercises or activities or even examples - it's definitely more of a "sit down and listen to what I have to say" approach to instruction. Ultimately I believe as a learner I would be incredibly bored with this textbook and would rather pay for a textbook that actually gives me more active learning content.
Consistency Rating: 2 out of 5
Q: The text is easily and readily divisible into smaller reading sections that can be assigned at different points within the course (i.e., enormous blocks of text without subheadings should be avoided). The text should not be overly self-referential, and should be easily reorganized and realigned with various subunits of a course without presenting much disruption to the reader.
Overall the text is broken up fairly well, there there are some points I noted in earlier comments where things didn't feel to be in the correct spot (e.g., putting qualitative research with quasi-experimental instead of with other non-experimental approaches). There are points in which there is self-referential statements, to give an example in the "Overview of Nonexperimental Research" (p. 117) it states "As we saw in Chapter 6, experimental research is appropriate..." - I often like to teach Nonexperimental methods first before going into experimental design - so these kinds of statements can be confusing to the students. That's just one example, I remember seeing numerous others in the textbook.
Modularity Rating: 3 out of 5
Q: The topics in the text are presented in a logical, clear fashion
Seems pretty in-keeping with a standard Research Methods textbook. Chapter 11 (presenting your research) however would generally make more sense if it followed the chapters on data analysis (e.g., as the last chapter in the book) - in practice you need to analyze your data first before presenting it. That would be the only comment on that.
Organization Rating: 4 out of 5
Q: The text is free of significant interface issues, including navigation problems, distortion of images/charts, and any other display features that may distract or confuse the reader
As noted - some of the images are oversized and make poor use of space (take up too much space given what they are depicting). The pictures on how to structure a poster (p. 213) just feel unnecessary to me, it would make more sense here to give an actual example of what a real research poster in Psychology looks like. Overall though, the interface is pretty good.
Interface Rating: 4 out of 5
Q: The text contains no grammatical errors
Overall, the text was pretty well edited, I did not some spots that were awkwardly worded, or featured run-on sentences. To give an example, on p. 165 in the "Conducting Surveys" one sentence states "The population might be all registered voters in British Columbia, all Canadian consumers who have purchased a car in the past year, women in the Quebec over 40 years old who have received a mammogram in the past decade, or all the alumni of a particular university." - I would take the "the" out of "in the Quebec" and shorten this sentence to improve clarity. There are other examples of small grammatical errors throughout - but nothing too overt.
Grammar Rating: 4 out of 5
Q: The text is not culturally insensitive or offensive in any way. It should make use of examples that are inclusive of a variety of races, ethnicities, and backgrounds
The textbook doesn't explicitly exclude anyone on the basis of culture, gender, ethnicity, national origin, age, disability, sexual orientation, education or religion - but it also doesn't do anything to include different perspectives or knowledge. The textbook for instance could take steps to revolutionize how we talk about research in Psychology by including a section on Indigenous Ways of Knowing. There was an outstanding article that was published by Lynn F. Lavallée on thus use of Indigenous Research Methods in answering questions important to psychology, (https://www.homelesshub.ca/sites/default/files/i2lyr5dn.pdf) and I would love to see textbooks open up more to moving outside of the traditional, materialistic, Westernized model of how research is conducted. These methods for instance would be perfectly suited to the chapter on Qualitative approaches to research.
Cultural Relevance Rating: 3 out of 5
Q: Are there any other comments you would like to make about this book, for example, its appropriateness in a Canadian context or specific updates you think need to be made?
No - I would not recommend it, for the following reasons.
1. It's out of date.
2. It's sparse and gives the students few extra resources, guidance, example activities or practice questions.
3. Some of the content is confusing (particularly as it relates to data analysis)
4. It doesn't have an Index or a Reference section
5. The visuals are pretty blah - not awful - but not exactly captivating either.
6. Some of the content (e.g., the discussion of validity) is over simplified or inaccurate.
I don't want this to come off as a major attack on this textbook - I just wanted to point these out in case there were questions as to why people might not be adopting this textbook anymore. I certainly wouldn't adopt this book over the paid book I'm using now (https://us.sagepub.com/en-us/nam/research-methods-for-the-behavioral-sciences/book259289) which comes in at a decent price point of about $70, is about the same size of book as the OpenEd book (and same level of difficulty), but is infinitely more accessible and more relevant in how it approaches these topics. I don't believe its a matter of simply "overhauling" the OpenEd book as many of the sections are out of date - so it would need to be largely re-written.