Using Altmetrics as a New Measure of Hospitality Management Faculty Productivity and Scholarly Impact

Lora Amsberryaugier and David M. Pearlman, University of New Orleans

Abstract
Quantifying faculty productivity has been of interest since the creation of the Journal Impact Factor.  With the electronic age, copious documents can be analyzed to calculate the h-index, which assesses the productivity and impact of the citation.  Changing generational characteristics have increased the use of third-party websites and their emerging role in disseminating research justify this pilot study examining the application of altmetrics along with h-index scores to assess Hospitality Management faculty productivity.  Findings corroborated the positive correlation between h-index and program recognition and altmetric score.  A range of penetration rates regarding altmetric events among hospitality management faculty was found.  
	
Key Words: faculty productivity, h-index, altmetrics, hospitality management

Using Altmetrics as a New Measure of Hospitality Management Faculty Productivity and Scholarly Impact
The cost of journal subscriptions and library materials continue to increase as library budgets remain stagnant or are cut.  In response, libraries cancel journal titles that decreases the number of full-text titles immediately available to their researchers. This new reality opens the door for librarians to advocate for open access and to demonstrate that peer review outside the traditional journal model are legitimate methods of scholarly communication and can improve scholarly communication within disciplines and even foster interdisciplinary research.  Piwowar et al. (2018) noted the growth of open access publishing, estimated to be 28% of scholarly literature, and our poor understanding on its potential impact on hiring and decision-making processes. Scholars seeking tenure and promotion are leery of open access because most do not understand how the peer review process works in open publishing models and how the impact of this work can be measured (Schimanski & Alperin, 2018).  This finding suggests the need to investigate the understanding and acceptance in using altmetrics to measure the reach and impact of an article outside the traditional peer-reviewed journal structure.
Schimanski and Alperin (2018) discussed whether the academic system and methods for evaluating faculty contributions has kept pace with technology and communications. Additionally, their synthesis of literature regarding review, promotion, and tenure (RPT) practices in the United States and Canada indicated that research and publications are presently the most important component of the review process and noted slow movement to replace journal impact factors with citation counts and altmetrics (Schimanski & Alperin, 2018).  Libraries have been the core proponents of open access and altmetrics. Since other disciplines have not embraced this method of scholarly communication and impact measurement, the authors wanted this research in an area of study outside of librarianship.  The purpose of this pilot study is to explore the application of altmetrics along with h-index (traditional) scores to assess Hospitality Management faculty productivity. The following pages outline faculty productivity assessment and then citation databases ending with a few research questions.   
Faculty Productivity
Faculty productivity reflect program strength and is an area that universities always strive to improve (Lee & Law, 2011).  One measure of a researcher’s achievement is by having a society changing impact such as developing a cure for a deadly disease or inventing an engine that is powered by trash; however, most researchers will not be that fortunate in their careers.  Cornell University established the first School of Hotel Administration in 1922, with the goal of studying the “science” of this work (Cornell University School of Hotel Administration, 2018) Outside the U.S. technical schools offered this curriculum until the mid-1960s when universities began adding hospitality education (Strauss, 2017). In the past, campus administrators assessed faculty productivity by graduates who could “read and write” i.e. knew the basics of their discipline; then, whether students could get a job after they graduated.  Faculty were expected to be knowledgeable, noted in their field, and contribute to the body of knowledge in their discipline following a tripartite mission of teaching, research, and service (Woods, 2006).  Fast forward to the 21st century, administrators measure faculty productivity by class sizes, student evaluation scores, and the quality and quantity of publications.  Blakey et al. (2017) found that the increase in research requirements was moving from the ground up, with faculty preferences leading hiring/tenure decisions affecting values, practices, policy, eventually becoming university standards/mission.  
Over the years, sophistication in counting publications and assessing publication quality has increased permitting the inclusion of new tools and measures to evaluate faculty productivity (Johanson & Woods, 2002; Jogaratnam et al., 2005; Lazaridis, 2010).  Chekitan et al. (2015) developed and proposed a new index, Dp2, to measure not only publications and citations but also consistency and longevity; however, this index only uses the traditional citation measurement. The fact that cites can take years to accumulate influences research productivity evaluation because it requires multiple years after publication to measure its importance. ISI (now Thomson Reuters) created the Journal Impact Factor (JIF) in the 1960s as a measure of quality to allow scholars to understand the value of content published in a journal relative to other journals representing the average number of citations within the previous two years creating pressure for faculty to publish in journals with high JIF scores (Konkiel, 2013).  
Since the 1980s, many questioned the supremacy of JIFs over citation counts.  With the advent of the internet, copious amounts of documents can be analyzed permitting the calculation of improved measures like the h-index.  The h-index, or Hirsch index, measures the impact of a particular scientist rather than a journal and "it is defined as the highest number of publications of a scientist that received h or more citations each while the other publications have not more than h citations each” (Schreiber, 2008, p. 1513).  For example, a researcher has an h-index of 10 if other authors cite 10 of their papers at least ten times.  The h-index is a method of measuring the productivity and impact of an academic’s work and has been used as a metric in the ranking of faculty and institutions (Lazaridis, 2010).  Many varieties of the h-index have been developed to address different purposes and idiosyncrasies (Roemer & Borchardt, 2012).  
Although scholars may traditionally have found articles by browsing journals, searching print indexes, attending meetings and checking correspondence with peers, in the era of digital sources they may rely upon keyword searches or online browsing instead.  Further, changing generational characteristics (digital natives) use of technology has changed the way we learn, work, and communicate.  These changes are an impetus to begin reevaluating traditional assessment criteria for scholarly impact.  Today’s scholars have many more options for collecting data, analyzing and disseminating their works with a growing number choosing open access.  This next generation researcher goes to the library less often, starts research using a search engine and not from a library's website, and has increased its use of third-party sites.  
Citation Databases
The citation databases listed below measure the traditional metrics and include Web of Science (WoS), Google Scholar (GS,), and Publish or Perish.  A presentation of altmetrics is also described to illustrate the increased benefit in utilizing nontraditional measures in faculty assessment.
Web of Science (WoS)
WoS is an online subscription-based scientific citation indexing service.  This bibliographic database is maintained by Thomson Reuters, providing a comprehensive citation search of journal articles, conference proceedings, abstracts, and other publication types.  It gives access to multiple databases that reference cross-disciplinary research allowing in-depth exploration of specialized sub-fields including the sciences, social sciences, arts, and humanities.  With over 90 million records covering 5,300 social science publications in 55 fields, 800 million+ cited references, and 8.2 million records across 160,000 conference proceedings (Web of Science, 2015).  However, it does have some limitations; namely, that it is not as extensive as Scopus with around 10,000 active journals, while it has better coverage of sciences than arts and humanities and has an English language bias.  Databases such as the Web of Science are selective in their journal coverage and provide better coverage of some disciplines in comparison to others.  Conference proceedings and monographs, which are a primary dissemination channel are not adequately covered (MyRI, 2011).
Google Scholar Citations
Google Scholar Citations summarize recent citations to many publications, to help authors consider where to publish their new research and provides an easy way for authors to gauge the visibility and influence of recent articles in scholarly publications quickly.  Google provides a five-year h-index and h-median scores that enable one to see which references were cited the most often and by who.  Google Scholar (2015) calculates an h-index which is not always the same as the WoS h-index rating because Google’s program is dependent on the researcher creating/monitoring their account.  If they do not create an account, they are not in Google Scholar.  Once a scholar registers the search engine will retrieve everything with the same name even if it is not them, which will skew their score.  It is up to the account holder to regularly review the references attributed to him/her and remove those that were written by someone else. The end user is not able to make corrections to the results, so if the scholar has a common name, the results should be scrutinized to determine if they are accurate.
Publish or Perish
Publish or Perish is a FREE software program that retrieves and analyzes academic citations.  It uses Google Scholar to obtain the raw data and then calculates a series of citation metrics.  The Author impact analysis page allows you to perform a quick analysis of the impact of an author's publications.  Publish or Perish uses these parameters to perform an Advanced Scholar Search query, which is then analyzed and converted to various statistics (Harzing-Publish or Perish, 2015).  
Altmetric Explorer
Altmetric Explorer is a subscription product, but access to its database is available for free to librarians. Altmetric.com's mission is to make article-level metrics easy.  It collects download statistics, reference manager counts, links from mainstream media sources, and social media shares and discussions, enriches the resulting data with demographics and profile information, and then makes everything available through APIs (Application Programming Interfaces) and for analysis through a web app called the Explorer.  Altmetric.com provides summaries of altmetric indicators and performs some cleaning and standardization of the data (e.g., by counting only the number of Tweets provided by unique Twitter users).  
In growing numbers, scholars are moving their work to the web.  Online reference managers Zotero and Mendeley each claim to store over 40 million articles, as many as a third of scholars are on Twitter, and a growing number tend scholarly blogs (Adie & Roe, 2013).  Adie & Roe (2013) said that these new forms reflect and transmit scholarly impact that once used to live on a shelf but now lives in Mendeley, CiteULike, or Zotero, where it can be seen and counted.  That hallway conversation about a recent finding has moved to blogs, and social networks and the local genomics dataset has moved to an online repository, both can be tracked.  Altmetrics expand our view of what impact looks like, but also of what is making the impact (Adie & Roe, 2013).  
Social media mentions, being available immediately after publication and even before release in some cases offer a more rapid assessment of research impact.  Social media sharing includes more than just research articles, it allows for measuring interest in book chapters, books, blog posts, and other forms of research that aren't part of traditional citation indexes. In contrast, citation indexes only assess the impact of scholarly literature on those who cite and neglects many other audiences of academic writing who may read but do not cite.  In particular, the societal implications of research may not be well addressed by citation indexes, and a range of alternative methods has been developed to assess this (Thelwall et al., 2013).  
Changing demographics are rewriting (read "posting") the way scholarship is being shared; to stay current, we must embrace these new communication channels.  Big data allows for all the metrics retrieved from the entire web to calculate non-traditional metrics proposed as an alternative to more traditional citation impact metrics, such as JIF and h-index.  Since we can now develop such information, we should at least learn of its application.  Priem et al. (2012) note growing numbers of scholars are integrating social media tools like blogs, Twitter, and Mendeley into their professional communications.  Metrics based on these activities could inform broader, faster measures of impact, complementing traditional citation metrics.  Correlation and factor analysis suggest citation and altmetrics indicators track related but distinct impacts, with neither able to describe the complete picture of scholarly use alone (Priem, Piwowar, & Hemminger, 2012).  Altmetrics are classified into five categories viewed (PDF downloads), discussed (Twitter), saved (Mendeley), cited (WoS), and recommended (used by F1000Prime).  
First proposed in 2010, the social web metrics are relatively young and (Priem et al. 2012) refer to mentions of scientific outputs in social web tools including Facebook, Twitter, blogs, news media or online reference management tools.  Altmetrics aim at complementing and improving the limitations of both traditional (i.e., Bibliometrics) and web-based (e.g., download and usage data) impact metrics and giving new insights into the analysis of impact.  The prevalent use of the social web by scholars has also led to some studies conducted on the analysis of altmetrics and its relation or association with previously established impact metrics such as citation analysis.  Most of these studies have found correlations (low, medium, and high) among altmetrics and citation scores suggesting that these two approaches are somehow related but that altmetrics might capture other types of an impact than citations (Priem et al., 2012).  Table 1 provides a listing of several identified Altmetric Events used to evaluate research contributions.

Table 1
Altmetric Events

Events	
WoS cites thru 2011	Facebook shares	
PDF downloads	Delicious Bookmarks	
HTML page views	Blog mentions	
WoS cites thru 2010	PLoS comment responses	
Scopus cites	PLoS comments	
CrossRef cites	Twitter mentions	
Mendeley saves	Facebook comments	
CiteULike saves	Facebook likes	
F1000 rating	Facebook Clicks	
Wikipedia cites		

Purpose of the Study
The purpose of this pilot study is to explore the application of altmetrics along with h-index scores to assess Hospitality Management faculty productivity.  Hospitality Management is a relatively new discipline, and by its nature, its faculty are likely to be interested in and use social media tools in their scholarship.  This research assessed three different methods/solutions for calculating the h-index and then generated altmetrics “scores” using Altmetrics Explorer.  As part of this pilot study, the researchers developed the following questions: 
1.	How does the penetration rate of hospitality and tourism faculty altmetrics compare between institutions?  
2.	Considering the increasing popularity of online information dissemination, are altmetrics necessary to thoroughly assess hospitality and tourism faculty contributions? 
3.	Considering the growing importance and benefits of an online presence to a program, how will altmetrics be utilized by faculty members, potential students & families, recruiters, and possibly the program ratings services such as U.S. News and World Reports?
Methods and Data Collection
Costas et al. (2014) noted that only 15%-24% of the publications presented some altimetric events although noted their presence was increasing.  Papers from the social sciences, humanities, and the medical and life sciences show the highest occurrence of altmetrics.  This paper was an exploratory study investigating the utility and benefit yielded from including altmetrics when analyzing Hospitality Management faculty productivity.  Further, h-indices were generated using three established sources/methods WoS, Google Scholar, and Publish or Perish to compare differences with ease of collection and accuracy of measurement.  
For this pilot study, the sample was selected based on the graduation rates and the top ten hospitality management programs were used for this research.  Thinking that these programs would most likely have the highest number of faculty publishing research with a higher likelihood of mentions on social media.  The National Center for Education Statistics, IPEDS Data Center is responsible for collecting and analyzing data related to education.  The study used the IPEDS 2012-2013 academic year data to identify hospitality management graduates by the institution (National Center for Education Statistics, 2014).  Only the faculty listed on departmental websites were counted for this study.  The numbers of graduates and faculty members of the ten largest Hospitality Management granting institutions are presented in Table 2.
Table 2
Pilot Study Top 10 Institutions with Highest Number of Graduates and Faculty Count

Ranking	Institution and Location	2012-2013 Graduates	Faculty	
1	University of Nevada-Las Vegas, NV	860	40	
2	University of Central Florida-Orlando, FL	820	38	
3	Florida International University-Miami, FL	771	25	
4	Johnson and Wales University-Providence, RI	411	2	
5	Culinary Institute of America-Hyde Park, NY	344	10	
6	University of Houston-Houston, TX	326	18	
7	Cornell University-Ithaca, NY	318	38	
8	California State Polytechnic University-Pomona, CA	282	17	
9	Northern Arizona University-Flagstaff, AZ	254	13	
10	Florida State University-Tallahassee, FL	218	9	

The h-index is an index that attempts to measure both the productivity and citation impact of the published body of work of a scientist.  The index measures the scientist's most cited papers and the number of citations that they have received in other publications.  The index is designed to improve upon more straightforward measures such as the total number of citations or publications.  As the alternatives have proliferated, comparative studies have become possible showing that most schemas correlate closely with the original h-index, but did not cut across disciplines, often requiring the evaluation of alternative indexes to decide between comparable CVs (Hirsch, 2005).
The Web of Science Core Collection database (1975–present) citation analysis tool calculated the h-index of the authors found in this database.  Each faculty member was searched in the database using the search methods required (Web of Science, 2015).   Additional limiters helped refine the search. "Social Sciences,” “business economics" and "social sciences special topics" narrowed the search results to authors publishing in the hospitality and tourism discipline.   Altmetrics tracks scholarly activity “mentions” on the open web and altmetric.com/explorer was used to generate this data.  Librarians can request and receive free access that allows them to search for article impact and to search for authors.  Altmetrics.com has a subscription product that it markets to employers where it builds a dashboard that tracks the social media mentions of all its researchers or faculty.  The data collection entailed entering each faculty member's name into the search box as a keyword search without permitting any further search refinement.  These results journal titles were reviewed to eliminated false hits.  Table 3 presents the h-index statistics for each institution and social media penetration rate of the institution’s faculty altmetrics.  
Table 3
h-index Statistics and Altmetrics Penetration Rates by Institution

Institution	

HRT
Faculty	Web of Science	Google Scholar	Publish
or
Perish	Altmetric.com	Altmetrics Penetration Rate Among Faculty	
		h-index	h-index	h-index	Sum of article scores, all-time average 2015		
Cornell University –
Ithaca, NY	38	4.5	5.947	12.579	3.421	26.3%	
University of Central Florida – Orlando, FL	38	2.211	3.947	4.684	0.711	42.1%	
University of Houston –
Houston, TX	18	1.889	3.111	6.222	0.444	27.8%	
Florida State University – Tallahassee, FL	9	1.778	0	2.111	0.667	11.1%	
University of Nevada –
Las Vegas, NV	40	1.7	1.325	6.625	0.325	27.5%	
Northern Arizona University – Flagstaff, AZ	13	0.941	1.615	1.923	0.077	7.7%	
California State Polytechnic University – Pomona, CA	17	0.471	0.588	1.118	1	17.7%	
Florida International University – Miami, FL	25	0.36	0.68	12	0.04	4.0%	
Culinary Institute of America – Hyde Park, NY	10	0	0	0	0.7	10.0%	
Johnson and Wales University – Providence, RI	2	0	0	0	0	0%	
* Faculty member must create an account for google scholar citation report to be found	
** Data source: Google Scholar	

The findings from this research corroborated with the literature regarding the positive correlation between h-index and program recognition and altmetric score (Konkiel, 2013; Roemer & Borchardt, 2012).  There was a positive correlation between the WoS h-index and the altmetric.com all-time average (r=.803, n=10, p=0.005).  The study calculated the penetration rate by dividing the faculty count from the total of all faculty with altmetric scores and findings indicated a range of penetration rates of altmetric events among hospitality management faculty from 0 to 42.1% indicating that some institutions are keen adopters of social media to disseminate their scholarship.  The same searches were conducted in both 2014 and 2016 to gauge ongoing social media interest and Table 4 shows this comparison.

Table 4
Ongoing Social Media Interest in Faculty Research
	
Institution	HRT Faculty	Altmetric.com
Sum of article scores, all-time average
2014	Altmetric.com
Sum of article scores, all-time average
2016	Percentage change in altmetric scores	
Cornell	38	3.421	11.5	30	
UCF	38	0.711	4.447	16	
UH	18	0.444	16.944	3	
FSU	9	0.667	3.222	21	
UNLV	40	0.325	2.55	13	
N. Arizona	13	0.077	0.385	20	
CA Poly	17	1	6.882	15	
FIU	25	0.04	0.94	4	
CIA	10	0.7	1	70	
Johnson and Wales	2	0	0	0	

Discussion of Results
Peer review is the most important means of ensuring quality scholarship traditionally presented in established journals.  Altmetrics offers a way for peer review to take place outside of the journal acceptance process.  Breaking away from this traditional process opens the door for peer assessment of non-print research outputs such as videos and blogs.  Altmetrics is tied to the idea of open access and sharing of information and research so that ideas and discoveries can be implemented to the benefit of society regardless of format.  Altmetrics take away the perceived value of the journal and focuses on the value of the article or individual research output.  Removing the focus on the journal allows research output to be evaluated without bias in favor of a journal with a high JIF score.  Upon completion, observations made suggest that collaboration with a librarian coauthor is critical for anyone undertaking citation research or any other library science-based endeavor.  For starters, the librarian ably demonstrated the lack of reliability of the Google Scholar's database in producing complete academic records because of its reliance on account holders maintaining their entries.   On the other hand, the Web of Science is 100% complete regarding its database; however, Web of Science limits itself by its selectivity of the journals it chooses to index.  
The pilot-test revealed several challenges to data collection.  First, access to all of the faculty members’ CVs would have simplified the process of verifying authorship of the works retrieved by the citation databases searched.  Second, the speed of data collection was enhanced if a middle initial is included in the search.  Web of Science indexes first and middle initials and not first names, so the middle initial is needed to differentiate common names from each other.  Coincidentally, in collecting the data for this study, two authors with the same two initials and same last name, at two different institutions research on similar, but different topics were identified and only after checking the department’s faculty roster at each institution was it confirmed that they were two separate people.  This coincidence demonstrates the attention to detail required to accurately conduct such analysis.
This study used the journal interface in Altmetrics Explorer to pinpoint hospitality and tourism articles more easily is offered free to librarians.  Altmetrics captures a broad aspect of research visibility and impact when compared to citation counts.  For example, non-publishing so-called "pure" readers are estimated to constitute one-third of the scientific community, and these may tweet or blog articles without ever citing them (Thelwall et al., 2013).  Thelwall et al. (2013) classified readers of scholarly publications into four groups: researchers, practitioners, undergraduates, and the interested public; also providing strong evidence that six of the eleven altmetrics (tweets, Facebook wall posts, research highlights, blog mentions, mainstream media mentions, and forum posts) were associated with citation counts for medical and biological sciences.  Although the results above suggest that altmetrics are related to citation counts, they might be able to capture the influence of scholarly publications on a more extensive and different section of their readership than citation counts, which reflect only the behavior of publishing authors (Thelwall et al., 2013).  Altmetrics can provide real evidence of public engagement with open access research outputs.
Conclusion
Quantifying faculty productivity has been of interest since the creation of the Journal Impact Factor.  With the electronic age, copious documents can be analyzed to calculate the h-index, which assesses the productivity and impact of the citation.  Nevertheless, changing generational characteristics have increased the use of third-party websites and their emerging role in disseminating research justify this pilot study examining the application of altmetrics to assess Hospitality Management faculty productivity.  Findings corroborated with the literature in revealing a positive correlation between h-index, program recognition, and altmetric score.  A range of penetration rates regarding altmetric events among hospitality management faculty was found.  
This pilot study revealed a few implications regarding the use of third-party websites to assess faculty productivity, methods for disseminating scholarship, and what channels for sharing scholarly work are considered acceptable.  Acceptance of alternative metrics will increase as researchers continue to evolve in how they create and share scholarship.  The review of the literature did yield a listing of benefits in collecting altmetrics including a measure of reach and influence across numerous dissemination channels.  Altmetrics permits the discovery of topics that are trending.  It permits the identification of potential collaborators for existing data, information to avoid prior mistakes, the discovery of alternative results or interpretations.   
As with the JIF scores, altmetric scores are not readily available to the general population.  They are tools promoted to libraries and campus leaders to evaluate faculty scholarship.  Until these tools are available on the open web, they will continue to have minimal impact on how the public perceives research as a recruitment factor in any but the highest funded institutions and departments.  However, the presence of research on the open web through social media platforms brings research to the general public that otherwise would stay within the realm of academia offers opportunities for program publicity, research sponsors, and generating student interest.
Study Limitations
NISO released altmetric standards after this study was conducted which may limit the utility of this study if these standards introduce major changes to the old metrics.  Also, in contrast to traditional metrics, Altmetric.com’s data is more robust for publications published from July 2011 onwards.  Google Scholar Citations has limitations because its reliability is dependent on the researcher to create and monitor their account which dramatically reduces its value as a research tool because the end user is not able to make corrections to the results.  For example, the results for a faculty member with a common name will need to be scrutinized for accuracy because the automated retrieval of citations will result in false hits on records not belonging to the researcher if they have not maintained their account.  Data gathering for this study found many cases with incorrect articles.  Another issue with Google is that not enough faculty have accounts and because of these shortcomings, this pilot study did not use Google Scholar Citation in the final data set for this research.
Future Research
Does it matter that you are not cited if you are read?  This question still needs to be answered and perhaps over time greater adoption of hospitality and tourism faculty will use non-traditional channels to disseminate their works.  How do these altmetric “counts” reflect on the quality of your program?  Again, it is too early in the adoption of social media to disseminate hospitality and tourism research to assess, however, there were a few professors and institutions that clearly see value in sharing their work in open source channels with high and increasing altmetric scores.  Future research is needed to assess whether high journal citation scores or even high altmetric scores equate to a high-quality program.  Research should be extended to more hospitality and tourism programs to ascertain what is considered “good” regarding altmetrics for HRT scholars.  The average h-index score for hospitality and tourism faculty by research subcategory ranged quite a bit regarding their altmetrics activity; however, future research should investigate characteristics of low and high institutions.  Researchers must ask if altmetrics reflect impact or just empty buzz.  Work comparing altmetrics with expert evaluation or traditional assessments like U.S. News and World Report and more specifically other Hospitality Program assessment study findings would be of interest.  Costas et al. (2014) maintain that one of the highest areas of altmetric activity is in the social sciences, but there is increasing activity in many areas of research.  Does this hold for hospitality research?  
This pilot study did not research what entity posted the research to the web or in what format, but anecdotally, other than CVs and researcher websites; publishers promoted much of the research in their journals. There was also some evidence in the larger programs that someone from within the college/department was charged to post when a faculty member had an article published. This study looked at each program collectively. Follow up research may want to focus on the attitudes and awareness of altmetrics by individual faculty at each institution. This type of study provides opportunities for librarians to collaborate with the college department to help disseminate its research. This work is also an opportunity for the library to aid in the promotion of the quality and value of the institution's research.
References

Adie, E., & Roe, W. (2013, January). Altmetric: Enriching scholarly content with Article-level discussion and metrics. Learned Publishing, 26(1), pp. 11-17.
Blakey, E., Khachikian, C., & Lemus, D. (2017). Increasing research requirements for tenure at teaching universities: Mission creep or mission critical? Teacher-Scholar: The Journal of the State Comprehensive University, 8(3). Retrieved from http://scholars.fhsu.edu/ts/vol8/iss1/3
Cornell University School of Hotel Administration (2018). Retrieved from sha.cornell.edu
Costas, R., Zahedi, Z., & Wouters, P. F. (2014). Do ‘altmetrics’ correlate with citations? Extensive comparison of altmetric indicators with citations. Netherlands: Centre for Science and Technology Studies.
Crisp, A. (2014, August). If you're not mobile, you're nowhere - GenerationWeb 2014. Retrieved from http://www.carringtoncrisp.com/blog/article/72/
Google Scholar. (2015). Google Scholar Metrics Overview. Retrieved from https://scholar.google.com/intl/en/scholar/metrics.html#overview
Harzing-Publish or Perish. (2015). Author impact analysis. Retrieved from http://www.harzing.com/pophelp/author-impact.htm
Hirsch, J. (2005). An index to quantify an individual’s scientific research output. Retrieved http://www.pnas.org/content/102/46/16569.abstract
Jogaratnam, G., Chon, K., McCleary, K., Mena, M., & Yoo, J. (2005). An analysis of institutional contributors to three major academic tourism journals: 1992-2001. Tourism Management(26), 641-648.
Johanson, M., & Woods, R. H. (2002). Analyzing U.S. Hospitality Faculty's Research Output. Journal of Hospitality & Tourism Education, 14(3), 52-56.
Konkiel, S. (2013, July/August). Altmetrics: A 21st-Century Solution to Determining Research Quality. Retrieved from http://www.infotoday.com/OnlineSearcher/Articles/Features/Altmetrics-A-stCentury-Solution-to-Determining-Research-Quality-90551.shtml
Lazaridis, T. (2010). Ranking university departments using the mean h-index. Scientometrics(82), 211-216.
Lee, A., & Law, R. (2011). Research productivity and institutional characteristics of hospitality and tourism programs. Journal of Travel & Tourism Marketing(28), 432-450.
Loria, P. (2011). The new metrics cannot be ignored – we need to implement centralised impact management systems to understand what these numbers mean. Retrieved from http://blogs.lse.ac.uk/impactofsocialsciences/2013/03/05/the-new-metrics-cannot-be-ignored/
MyRI. (2011, April 17). Calculating the h-index: Web of Science, Scopus or Google Scholar? Retrieved from http://www.ndlr.ie/myri/
National Center for Education Statistics. (2014, December 2). National Center for Education Statistics. Retrieved from http://nces.ed.gov/ipeds/datacenter/InstitutionByName.aspx?stepId=1
Park, K., Phillips, W. J., Canter, D. D., & Abbott, J. (2011). Hospitality and tourism research rankings by author, university, and country using six major journals: The first decade of the new millennium. Journal of Hospitality & Tourism Research, 35(3), 381-416.
Pearlman, D., Ryu, K., & Schaffer, J. D. (2010). Assessing hospitality programs using objective criteria. Journal of Teaching in Travel & Tourism, 124, 103-124.
Piwowar H, P. J. (2018). The state of OA: A large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6(e4375). Retrieved from https://peerj.com/articles/4375.pdf
Priem, J., Piwowar, H. A., & Hemminger, B. M. (2012, March 20). Altmetrics in the wild: Using social media to explore scholarly impact. Retrieved from http://arxiv.org/abs/1203.4745
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2011, September 28). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto/
Roemer, R. C., & Borchardt, R. (2012). From bibliometrics to altmetrics: A changing scholarly landscape. Retrieved from http://crln.acrl.org/content/73/10/596.full
Schimanski, L., & Alperin, J. (2018). The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future [version 1; referees: 1 approved). F1000Research 2018, 7(1605), 1-19. Retrieved from https://doi.org/10.12688/f1000research.16493.1
Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513-1522.
Strauss, P. (2017). Caught between two stools? Academic writing in 'new' vacational disciplines in higher education. Teaching in Higher Education, 925-939.
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. (2013, May). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), 1-7.
Web of Science. (2015). Fact: Unrivaled coverage of subject-specific content. Retrieved fromhttp://wokinfo.com/citationconnection/
Woods, D. R. (2006, Fall). A case for revisiting tenure requirements. Thought & Action, 135-142.