Reflections on the prospects for evaluation and qualified production of graduate programs in Environmental Sciences

In this editorial, we reflect on the evaluation criteria that is under discussion to be adopted for the assessment of CAPES (Brazilian Federal Agency for the Support and Evaluation of Graduate Education) Environmental Sciences area – CACiamb. This criteria aims to increase qualified production by simulating CAPES quality strata A1, A2, B1 and B2 production of Academic Master Degree Programs with 12 permanent professors based on the criteria established by the Interdisciplinary Area Committee – CAInter in 2007. As well as expectations for the adoption of periodic assessment of free access bases, along with fostering the use of scientific journals published online by Graduate Programs.


INTRODUCTION
The Graduate Program Evaluation System was deployed by the Coordination for the Improvement of Higher Education Personnel -CAPES in 1976(CAPES, 2012a).Its Interdisciplinary Area (CAInter), in 2006, was organized into four Thematic Chambers: I -Environment and Agriculture; II -Social and Humanities; III -Engineering, Technology and Management; and IV -Biology & Medicine.Recognizing the complexity of environmental science, CAPES created, by the Ordinance 081 of June 6, 2011 (CAPES, 2011), the Area of Environmental Sciences -CACIAmb in Thematic Chamber I, that was effectively installed in the 1 st ReNaCiamb (CAPES, 2012c).
All graduate programs have to submit information annually to CAPES using the Data Collection Platform.This information, once treated and consisted, allows the issuance of reports used in the evaluation process.Thus, CAPES makes public on the web a page containing a set of 11 reports for each program: 1. Teses and Dissertations;2. Bibliographic Production;3. Technical Production;4. Artistic Production;5. Faculty, Enrollment and Graduation;6. Disciplines;7. Research Thematic Focus;8. Research Project;9. Proposed Program;10. Teaching Performance;and 11. Faculty Production (CAPES, 2012b).
The evaluation form consists of program indicators with different weights according to the degree of their importance.The main indicators measure the quality of both faculty and student participation in the activities of the Post-Graduate and related Undergraduate programs.The major indicator in most cases is the number of permanent faculty members.CAPES establishes the need for a strong and solid ties of the institution's faculty to the program, because production of collaborative teaching staff is counted only when there is joint production with permanent faculty members and students.
The 1 st ReNaCiamb (CAPES, 2012c) rose in the Environmental Sciences Area, the need to broaden the discussion of the evaluation procedure for this new Post-Graduation area to allow dialogue and community participation in the definition of parameters and criteria.However, as a first step, it is very natural and appropriate that the criteria previously used by the interdisciplinary area would be also used for the new environmental area, and particularities of this new area are to be incorporated progressively as discussions are settled.
The evaluation criteria for the Interdisciplinary Program Area (CAInter) are given in Table 1 ( CAPES, 2009).It can be observed that for a program with grade 3 to improve its grade, criterion 3 (Student Body, Thesis and Dissertations) and 4 (Intellectual Production) must reach at least the "good" concept.The major difficulty faced by the programs refers to the joint qualified production of faculty and students, especially in scientific journals.Some of the reasons that have been reported by the coordinators of the programs are: a) lack of perseverance of the graduated students to follow the later stages after an article was submitted to a journal; b) non-academic professional performance of graduates hampers the co-authored production; and, c) the long time between submission and publication in a qualified scientific journal.
We emphasize the importance of the criteria 3 and 4, both with 35% weight in the program evaluation concept (Table 1).Within criterion 3, Student Body, Thesis and Dissertation, the item 3.3, that refers to the quality of thesis and the production of graduate and undergraduate students authorship, has 50% weight in this criterion and reflects the measure of the proportion of students who are authors of full articles in relevant scientific journals and conferences, as well as books and book chapters and technological production based on the relevant indicators IndAut and IndQual (CAPES, 2009).
The IndAut is a relative measure of the number of postgraduate students that are authors of full articles in journals or annals of scientific meetings, of relevant books or book chapters, and of relevant technological/artistic production.While the IndQual is a relative measure of average intellectual production of the permanent professor staff with the participation of graduate students, including those graduated up to three years after graduation.
The fourth criterion, Intellectual Production (Table 1), includes an item 4.1.that refers to qualified publications by the permanent faculty members with a weight of 55% and it is measured by the IndProd (Equation 1) that reflects the qualified production in journal articles, books, book chapters and of technology products by the permanent teaching staff of the program (CAPES, 2009).There is a clear induction by CAPES to foster joint publications in high qualified journals.This brings a lot of pressure and responsibility on professors and students, as already stated by Silva (2009).The performance indicator of the average annual scientific production of the Interdisciplinary Area for academic post-graduate programs is shown in Table 2, which clearly spells out the minimum scores to achieve an attribute concept.The scores have to be distributed to all permanent professors.

Attribute
Production Indicator (IndProd) annual average A simulation of production score values in journals with CAPES Qualis A1, A2, B1 and B2 using Equation 1 to calculate the IndProd for a Master Degree Academic Program that has 12 teachers is presented in Table 3.It can be observed that if they produce only 12 Articles in one year, the professor production score, IndProd, will be equal to 0.59 and it will not pass the Regular (R) concept, even if the production concentrates in the most qualified journals.Thus TARGA, M. S. Reflections on the prospects for evaluation and qualified production of graduate programs in the Environmental Sciences.Ambi-Agua, Taubaté, v. 7, n. 1, p. 4-8, 2012.(http://dx.doi.org/10.4136/ambiagua.879)7 it is clear the need to increase production to a minimum of two articles per each member professor, in the same extracts, in order for the IndProd to achieve a Good (G) concept, for a Very Good (VG) concept, the production of the program would have to be tripled.The Qualis System was designed to improve the process of journal classification, so far lacking systematization and produced by artisanal and non-uniform means (Souza and Paula, 2002).In this sense, the evaluation criteria, proposed in the 1 st ReNaCiamb for Qualis Scientific Periodicals, was to take into account A1 to B5 layers and JCR and SciELO indexed journals, and other indexing databases such as BIOISIS, CAB, EconLit, FSTA, GEOREF, INDEX-PSI, LILACS, MATHSCI, MEDLINE, MLA, Philosopher, PYSCINFO, PUBMED, and SPORT DISCUS.It takes into account also if the journal is in the CAPES portal, if it has qualified and recognized editorial board, well established peer review procedure, and if it is edited by a scientific society, university or a research institution (CAPES, 2011).

FINAL REMARKS
Most indexing databases in adoption by CACiamb (environment), along with CAInter (interdisciplinary), emphasize only the Health area and, therefore, do not adequately cover the entire subject of most graduate courses that are grouped in his new area, which focuses in more comprehensive environmental research than just Health.Thus, the use of indices with greater scope and preferably freely accessible bases is recommended, as they would allow the analysis of all journals in an equal basis without leaving out emerging journals (Dias and Batista, 2010).
Online publication of scientific journals should be encouraged by CAPES, because they reduce the time for publication and have additional benefits when the access is free, such as facilitating research findings, maximizing the interaction between research groups, optimizing efforts, and increasing the efficiency of research funding, in addition to the improved visibility and increase in the number of citations, as previously observed (Lawrence, 2001;Batista, 2007).

Table 1 .
Summary of program evaluation criteria of CAPES Interdisciplinary Area.

Table 2 .
Performance indicators of the average annual scientific production of Interdisciplinary Area for academic post-graduate programs.

Table 3 .
Simulation Values of IndProd for an academic master's program.