Supplemental datasets for examination of the PCAOB's relationship with the audit profession in the United States

These datasets have been co-submitted to this journal with the research article “A multi-method analysis of the PCAOB's relationship with the audit profession” [1]. The purpose of these datasets is to assist readers of the research article in obtaining further details on two analyses performed therein: a deviant case analysis of the interview data (enclosed here as Dataset 1) and content analysis of disagreement-report pairs (enclosed here as Dataset 2). These datasets could also inform future research into audit regulation by serving as a starting point in developing research ideas or as a context to settings chosen by the researchers for examination. We developed Dataset 1 by examining interviews of eight PCAOB inspectors and six audit partners who were involved in early interactions between the largest firms and the PCAOB for elements of the data that are inconsistent with the explanations and results in the research article. We developed Dataset 2 by identifying audit firm response letters during the period of 2004-2012 where there are one or more instances of explicit disagreement with the PCAOB's inspection results within the response letter. We identified 15 such audit firm response letters and paired them with the subsequent inspection report based on an inspection that started after the audit firm response letter date. We then examined the content of disagreements and the subsequent deficiencies and highlighted issues in these subsequent reports that appear like disagreements.


a b s t r a c t
These datasets have been co-submitted to this journal with the research article "A multi-method analysis of the PCAOB's relationship with the audit profession" [1] . The purpose of these datasets is to assist readers of the research article in obtaining further details on two analyses performed therein: a deviant case analysis of the interview data (enclosed here as Dataset 1) and content analysis of disagreement-report pairs (enclosed here as Dataset 2). These datasets could also inform future research into audit regulation by serving as a starting point in developing research ideas or as a context to settings chosen by the researchers for examination. We developed Dataset 1 by examining interviews of eight PCAOB inspectors and six audit partners who were involved in early interactions between the largest firms and the PCAOB for elements of the data that are inconsistent with the explanations and results in the research article. We developed Dataset 2 by identifying audit firm response letters during the period of 2004-2012 where there are one or more instances of explicit disagreement with the PCAOB's inspection results within the response letter. We identified 15 such audit firm response letters and paired them with the subsequent in-spection report based on an inspection that started after the audit firm response letter date. We then examined the content of disagreements and the subsequent deficiencies and highlighted issues in these subsequent reports that appear like disagreements.
© 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license.

Specifications table
Subject Accounting Specific subject area Regulation of public company auditing in the United States Type of data Memo Table  How data were acquired Textual analysis of interviews of PCAOB inspectors and audit firm partners. Textual analysis of audit firm response letters and subsequent PCAOB inspection reports.

Data format
Raw Analysis

Parameters for data collection
For Dataset 1, we compiled data by examining confidential interviews of PCAOB inspectors and audit firm partners. For Dataset 2, we compiled data by examining publicly available audit firm letters and subsequent PCAOB inspection reports.

Description of data collection
For Dataset 1, we examined interviews of eight PCAOB inspectors and six audit partners who were involved in early interactions between the largest firms and the PCAOB for elements of the data that are inconsistent with the explanations and results in the research article. We developed Dataset 2 by identifying audit firm response letters during the period of 2004-2012 where there are one or more instances of explicit disagreement with the PCAOB's inspection results within the response letter. We identified 15 such audit firm response letters and paired them with the subsequent inspection report based on an inspection that started after the audit firm response letter date. We then examined the content of disagreements and the subsequent deficiencies and highlighted issues in these subsequent reports that appear similar to prior disagreements.

Value of the Data
These data provide supplementary materials we mention in our Accounting, Organizations, and Society (AOS) article titled "A multi-method analysis of the PCAOB's relationship with the audit profession". Readers of our AOS article, as well as researchers on regulation, could benefit from these data. These datasets can inform future research on regulation by providing materials to research, as well as insights that may be useful in developing research ideas and performing analysis.

Data Description
Dataset 1 contains deviant case analysis from interviews of eight PCAOB inspectors and six audit partners in relation to the related research article. The research article uses Responsive Regulation theory [2] as lens in developing empirical hypotheses and interpreting interviewbased insights. The deviant case analysis examines the points in our interview data that are inconsistent with the explanations in the research article.
Dataset 2 contains firm-PCAOB disagreement-report pairs during the period of 2004-2012 that we identified in two stages. First, we identified audit firm response letters where there are one or more instances of explicit disagreement with the PCAOB's inspection results within the response letter. We identified 15 such audit firm response letters and paired them with the subsequent inspection report based on an inspection that started after the audit firm response letter date. We then examined the content of disagreements and the subsequent deficiencies and highlighted issues in these subsequent reports that appear similar to disagreements.

Experimental Design, Materials, and Methods
In compiling Dataset 1, deviant case analysis, we followed the recommendation of Malsch and Salterio [3] on how authors can increase trustworthiness of their qualitative findings ( emphasis added): "Trustworthiness of the findings can also increase considerably when the researcher demonstrates they have carried out deviant or negative case analysis. This approach involves searching for, analyzing, and reporting on elements of the data that do not support, or appear to contradict, patterns of explanations that are emerging from data analysis (Silverman 2005)." (p. 13) One of the coauthors performed the deviant case analysis and two other coauthors independently reviewed the data and then the analysis to ensure that the analysis is thorough and includes all deviations.
In compiling Dataset 2, firm-PCAOB disagreement-report pairs during the period of 2004-2012, we performed analysis in two stages. First, we identified audit firm response letters where there are one or more instances of explicit disagreement with the PCAOB's inspection results within the response letter. We identified 15 such audit firm response letters and paired them with the subsequent inspection report based on an inspection that started after the audit firm response letter date. We then examined the content of disagreements and the subsequent deficiencies and highlighted issues in these subsequent reports that appear similar to disagreements. One of the coauthors performed this analysis and two other coauthors independently reviewed the data and then the results of this analysis to ensure that the analysis is thorough, includes all such disagreement-report pairs, and contains accurate descriptions.

Disclaimer
The views expressed are purely those of the authors and may not in any circumstances be regarded as stating an official position of the PCAOB or any of the audit firms involved.

Deviant Case #1 (Anonymous Partner):
Two of the partners believe that there was also ego involved at the PCAOB in response to the firms' disagreements and that certain lead inspectors took disagreements a lot more personally than others. One of these two partners believes that this was the main driver of the PCAOB's response to the disagreements, while the other partner believes that this was only another factor in addition to risk signaling. Because these are minority views among the partners and the inspectors whom we interviewed, we conclude that this is a deviation that does not materially affect our findings.

Deviant Case #2 (Anonymous Partner):
In one instance a partner responded to a question about whether the PCAOB used prior inspection reports in their subsequent inspections in the negative. The following was the specific conversation that took place: Interviewer: ... How did they zero in on areas where they thought, "You know what, we are most likely to find something there." Would they use the prior reports you think?
AP: No, no, no, no. How that worked was ... We know how that works. They all talk ... I don't know if they had a conference call every morning. I'm kidding okay? They talked to all the other teams. They knew exactly where to look.
However, it appears that this response was due to a misunderstanding of the question. As the topic developed further in the interview, it was clear that this partner believed that the PCAOB viewed the firms' public disagreements as areas of risk to focus on in the subsequent inspections, consistent with our theory. For example, the following subsequent dialogue took place: Interviewer: …you think they may have been taking some clues from what you [publicly] disagreed about? AP: Absolutely because let's say you have a professional disagreement over ... Take the [area] issue... We looked at it pretty good… Gave it a look right before we let the reports go out the door. Then you have ... a truly professional disagreement. …What do you do if you truly believe that you have followed the professional guidance and your firm's guidance, which is pretty detailed? It's not like they come in and say, "Well, you didn't follow this part of it." No. You did exactly what the firm told you to do, which was more than what the AICPA auditing standards had to do. You still got written up. If you don't think that'll tick some people off [resulting in a disagreement]? …That becomes one of their ... Okay, they gonna do an inspection, quasi audit. They identify their risk areas. What's their risk area? [A confidential issue X related to an audit standard]. The first thing they want to see [during the next inspection] is "Give me all your [workpapers related to issue X]." Thus, we conclude that the partner's initial response does not affect our findings because it was due to a misunderstanding of the question and does not reflect this partner's views on whether the PCAOB used public disagreements as signals of risk to focus on during subsequent inspections.

Deviant Case #3 (Anonymous Partner):
One partner expressed a view about why the firms ultimately stopped disagreeing publicly that was deviant from all other partners. Specifically, we documented the following as part of our summary of the interview with this partner: This partner mentioned that in the beginning, there was a lot of uncertainty. How will the markets view the reports? Were they going to be a scorecard? After a few years, it became apparent that the market did not care that much about the inspection report. Therefore, there was less of a need to defend the firm's reputation.
In addition, the partner mentioned that there were too many inspection findings mostly related to internal controls around 2010, so it became impractical to make statements about individual deficiencies.
However, in the second part of the interview the partner conveyed that (s)he definitely believes that the PCAOB used disagreements as risk indicators for subsequent inspections. Thus, except for the deviant view of why the firms stopped disagreeing publicly, this partner's views are consistent with our overall findings. Since this partner's view deviates from the rest of our data, we conclude that this view is a minor deviation that does not affect our findings.

Deviant Case #4 (Anonymous Partner):
One partner noted that from the beginning (s)he accepted the PCAOB as a regulator and did not seem to resent the PCAOB's punitive approach to regulation: "I'd say our firm's view was we can disagree politely. We need to remember they are the regulator." This partner further said: "We tried to be professional, cordial, respectful and always put in some platitudes around this is good for us and you're doing an important function. And we hope we all improve, getting better going forward. You and us." Thus, at first, this partner's position seems to contradict our overall findings that the firms resented the PCAOB's punitive approach and resisted its authority.
However, this partner clarified her/his position throughout the interview by explaining that all of the firms misjudged the PCAOB. Moreover, the partner explained that her/his position was somewhat different from the other firms' leaders in that (s)he saw the need to disagree with the PCAOB and to defend the firm's positions, but wanted to do this more diplomatically than some of the other Big 4 firms: So early responses [the firms] were first trying to formulate… my thought was we don't need to irritate [the PCAOB] needlessly. But my own personal view… was that we also had to stick up for our partners and our teams that did audits. And if we did not agree then we needed to say so. We shouldn't have [our partners] end up explaining that their audit was deemed an audit failure if all the way up the line [the firm] didn't think it was. …Again I don't say this to sound altruistic or anything, but it was important. If we want people to grow up wanting to be audit partners and auditors, which is maybe the safest profession one could choose, we had to back them at the same time [by disagreeing with the PCAOB if necessary].
Thus, given the partner's clarification of her/his nuanced position on disagreeing with the PCAOB, this partner's attitude toward the PCAOB appears to be nuanced-the partner did believe that the firms needed to disagree with the PCAOB at times, consistent with our reasoning. Thus, we conclude that overall this partner's view is nuanced, but entirely consistent with our theory.

Deviant Case #5 (Anonymous Partner):
One partner said the following: "I think a firm's public disagreement to an inspection report was, I won't say completely irrelevant, but couldn't have been a big part of it. There were just too many other things I think that were causing them to say… here's what they think we're doing, or should be doing." However, in the immediately following sentences the partner clarified "if any one firm was too loud in their response, they probably got a little nod [from the PCAOB] of you know, you're not supposed to be quite that demonstrative [laughs]. … I just don't think that really affected that much, I think it was more all the other things going on that probably drove [the PCAOB inspections], more than that. But again, I would not be telling the truth, there clearly were times that you certainly got the message. We are the regulators, and you are being regulated. And if our teams, our inspection teams, have a judgment about whether you did enough work on the [confidential issue X related to a financial accounting standard], and we think you should have done more, and here's what you should have done, that's the answer, not what you guys think. That at times was the attitude." Overall, this partner indicated that (s)he definitely believes that the PCAOB viewed disagreements as risk signals, but that from her/his standpoint there were also major other issues going on that affected the PCAOB's inspections. This partner also acknowledged that some other firms felt that disagreeing with the PCAOB affected them significantly, as this partner explained her/his view on why the firms stopped disagreeing: I think [the other Big 4 firms] just got the view that it might be construed [by the PCAOB] that the firm doesn't really get it and is not really taking the necessary steps to correct deficiencies if what they're really doing is arguing that they don't agree that there are deficiencies that need to be fixed. And so part of the constructive goal if you want to get better, you're saying you want to get better, we [the PCAOB] are telling you how to get better, and you're telling us we [the PCAOB] don't have a basis for that. I think that's really more of where the dialogue came in. And I'd say there were conversations that one could get the drift. There were probably a few things in writing on just how you process all these reports, but I think the firms got the hint.
Thus, overall, this partner's view is that public disagreements were consequential, but with the nuance that there were other more consequential factors that affected the dynamic between the PCAOB and the firms. This view when taken in its entirety is largely consistent with our theory and does not change our conclusions.

Deviant Case #6 (Anonymous Inspector):
One inspector had a view that the PCAOB's objective is to identify the right answer for the investors. Initially, this view seems to be more consistent with the persuasion-or cooperationbased approach than with punishment-based approach to regulation: The biggest thing is getting back to just focusing on the investor, and it is ... Are we getting to the right answer for the investor, and is that investor being protected? That was something you didn't hear at [a Big 4 where this inspector previously worked], at least I didn't. There was a lot around the budget, meeting the budget, collecting the audit fees. So, it was refreshing to get back, and we don't care how long something takes, we just care [about] getting to the right answer… Again, it's all geared towards getting to the right answer, so you're evaluated [as an inspector on] whether or not you're able to get to the right answer. …It's a totally different mindset from the firms, for sure… I remember when I was at the firm, right, it was always PCAOB's the evil company coming in, and be careful, and they're making our lives so difficult. Then I come on the PCAOB and I'm like, yeah, we really don't have an agenda when we go visit an office.
Upon closer analysis, this inspector's view is not inconsistent with our theory because the inspector still believes that it is the PCAOB who has the right answer, not the firms. The inspector also expressed frustration with the firms' lack of willingness to accept the PCAOB as a regulator who seeks to come up with the right answer. In this inspector's view, the partners lacked this willingness because they did not like being challenged. Overall, though this inspector has a softer take on the PCAOB's mindset, this inspector's views are not inconsistent with our theory. Thus, we conclude that this inspector's softer mindset is a minor deviation that does not affect our conclusions materially.

Deviant Case #7 (Anonymous Inspector):
One of the inspectors shared his/her opinion that one of the reasons the firm stopped disagreeing with the PCAOB publicly is because they realized that publicly fighting the PCAOB makes them look bad to the public and that the authority-based relationship with the PCAOB, where the firms accept the PCAOB's requirements, actually helps them increase their audit fees. At first, this specific view may seem inconsistent with our theory that the firms stopped because they recognized that the PCAOB will punish them for public disagreements. However, after a deeper consideration of this view, we concluded that this view is still substantively consistent with our theory in that the inspector believes that the firms stopped disagreeing publicly because they realized that they cannot change the PCAOB's conclusions by publicly disagreeing, thereby making themselves look bad to the public. The only inconsistent part is that according to this inspector the firms may have done so to repair reputation and to retain the increased fees. Thus, this is an interesting nuance, but one that is inconsistent with the other inspectors' views. Therefore, we conclude that this deviant case does not materially affect our conclusions.

Deviant Case #8 (Anonymous Inspector):
One PCAOB inspector stated toward the beginning of the interview that (s)he does not think that the disagreements affected what the PCAOB inspectors did. However, deeper analysis of this inspector's statement led us to conclude that the inspector merely wanted to state his/her position that the firms' disagreements do not bias the PCAOB inspectors' judgment toward a retaliation against the disagreeing firms.
In addition, the inspector immediately clarified his/her position on disagreements by explaining that "if a regulator says this isn't right and then a regulatee says 'no I disagree with you and I am not going to fix it' it may have us concentrate even more on this area. This could elevate the concern. Not in retaliation, but more as a normal reaction. If the regulatee says I disagreethis could be a bigger problem we had thought. They are telling us that they are not going to fix it because they think they are doing it correctly. Maybe a planning team thought 'Geesh they don't think this is a problem, we better focus on this area.' " Thus, though this inspector's initial statement appeared to deviate from our theory, the inspector subsequently elaborated on the statement and clarified his/her position. The clarified position of this inspector is consistent with our theory. Thus, this case does not appear to be inconsistent with our conclusions.

Overall Conclusion:
We identified each case (or element) within our data where the statement or a series of statements by the interviewees appear to be inconsistent with our theory. We then performed a deeper analysis and concluded that each deviant case is either a minor deviation, a nuance that is consistent with our theory, occurred due to the interviewee's misunderstanding of our question and was later clarified, or a statement that was vague and was later clarified by the interviewee. Overall, we conclude based on our deviant case analysis following the suggestion in Malsch and Salterio [3] that our conclusions reflect our data fairly.

Dataset 2: Content Analysis of Disagreement-Report Pairs
Emphasis is added to indicate issues in the subsequent reports that appear similar to prior disagreements. In this subsequent report, the PCAOB reported a deficiency related to insufficient going concern considerations. By comparison, the PCAOB did not report any deficiencies related to going concern in the Deloitte report issued on