After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 764515 - doxygen crashes no resolved
doxygen crashes no resolved
Status: RESOLVED FIXED
Product: doxygen
Classification: Other
Component: general
unspecified
Other Linux
: Normal blocker
: ---
Assigned To: Dimitri van Heesch
Dimitri van Heesch
Depends on:
Blocks:
 
 
Reported: 2016-04-02 20:52 UTC by Kemin Zhou
Modified: 2016-09-05 13:45 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Kemin Zhou 2016-04-02 20:52:54 UTC
When running the doxygen command, it crashed with version 1.8.10, 1.8.11
but not with 1.61
I have a mixture of C++ and Java files. It crashed on one of the Java files
This is reproducible on different platforms. So it is doxygen specific.


Parsing file /path/to/my/source/LoadCDSFromAce.java...
Segmentation fault: 11

I looked into this file and could not find anything special. This is an really old piece of code that has always been able to pass doxygen without any problem.
Comment 1 Kemin Zhou 2016-04-03 04:35:26 UTC
The directory is mostly C++ file. The first Java files are fine, the third one got segmentation fault.  I went through every single line of this crashed file.  Everything is normal. I cannot see any abnormal thing in this file.
Comment 2 albert 2016-04-03 07:50:40 UTC
Can you build doxygen yourself in debug mode and see with gdb where it crashes otherwise can you please attach a self-contained example (source+config file in a tar or zip) that allows us to reproduce the problem?
Comment 3 Kemin Zhou 2016-04-03 18:21:15 UTC
I was planning to do debug run because it is very difficult to reproduce on your end. Even on my end it may be a special combination of things.  I never had any trouble with doxygen before.
Comment 4 Kemin Zhou 2016-04-03 19:32:46 UTC
Parsing file /home/myhome/proj/ortho/LoadCDSFromAce.java...

Program received signal SIGSEGV, Segmentation fault.
0x0000000000683852 in QCString::StringRep::length (this=0x50)
    at /home/myhome/Downloads/doxygen-1.8.11/qtools/qcstring.h:581
581	          uint l = u.s.isShort ? u.s.len : u.l.d->len;
(gdb) bt
  • #0 QCString::StringRep::length
    at /home/myhome/Downloads/doxygen-1.8.11/qtools/qcstring.h line 581
  • #1 QCString::length
    at /home/myhome/Downloads/doxygen-1.8.11/qtools/qcstring.h line 197
  • #2 QCString::operator+=
    at /home/myhome/Downloads/doxygen-1.8.11/qtools/qcstring.h line 309
  • #3 scannerYYlex
    at scanner.l line 2052
  • #4 parseMain
    at scanner.l line 6922
  • #5 CLanguageScanner::parseInput
    at scanner.l line 7051
  • #6 parseFile
    at /home/myhome/Downloads/doxygen-1.8.11/src/doxygen.cpp line 9378
  • #7 parseFiles
    at /home/myhome/Downloads/doxygen-1.8.11/src/doxygen.cpp line 9470
  • #8 parseInput
    at /home/myhome/Downloads/doxygen-1.8.11/src/doxygen.cpp line 11093
  • #9 main
    at /home/myhome/Downloads/doxygen-1.8.11/src/main.cpp line 37

================= the content of the really old and useless java file: LoadCDSFromAce.java ==============
import acedb.*;
import java.util.*;
import java.sql.*;
import java.io.*;
import DBinfo;

/**
 * this class loads both species and CDS table from the component
 * Acedbs to the single Postgres DB
 * species must be loaded first because cds source is a foreign key
 */
public class LoadCDSFromAce extends DBinfo {
   private Ace aceDB = null;
   /**  
    * postgres DB connection
    */
   private Connection psconn; 
   /**
    * to query the PG db
    */
   private Statement stmt;
   private static PreparedStatement orgidStmt;
   /**
    * loaded in constructor
    */
   private Vector unusedIds;
   public static final String queryCDS = "select s, s->corresponding_protein[1], len, ss->Species[1] from s in class sequence, len in (s->CDS[2]-s->CDS[1]+1), ss in s->Source where exists_tag s->CDS and len > 18 and not exists_tag s->cDNA_EST";
   public static final String querymRNA = "select s, s->corresponding_protein[1], len, s->Species[1] from s in class Sequence, len in (s->CDS[2]-s->CDS[1]+1) where exists_tag s->CDS and len > 18 and exists_tag s->cDNA_EST";
   /**
    * unused id index
    */
   private int unusedIndex=0;
   /** 
    * for keeping track of the ids
    */
   private static int cds_id=0;
   /** 
    * set a limit on ace names
    */
   private static final int ACE_ID_LEN = 34;

   public LoadCDSFromAce(String pgDB, String usr) 
      throws SQLException, ClassNotFoundException
   {
      user = usr;
      this.pgDB = pgDB;
      Class.forName("org.postgresql.Driver");
      psconn = DriverManager.getConnection(pgDB, usr, pgPassword);
      stmt = psconn.createStatement();
      orgidStmt = psconn.prepareStatement("Select id from species where name=?");
      unusedIds=getUnusedId();
   }

   /**
    * no doc yet
    */
   public LoadCDSFromAce() throws SQLException, ClassNotFoundException {
      Class.forName("org.postgresql.Driver");
      psconn = DriverManager.getConnection(pgDB, user, pgPassword);
      stmt = psconn.createStatement();
      orgidStmt = psconn.prepareStatement("Select id from species where name=?");
      unusedIds=getUnusedId();
   }

   /**
    * This is to connect to an ACEDB on host h and port p.
    */ 
   public void connect(String h, int p) {
      aceDB = new Ace(h, p, user, acePassword);
   }

   public void connectAce(String dbname) {
      aceDB = createAce(dbname);
   }

   public void close() throws SQLException {
      orgidStmt.close();
      psconn.close();
   }
   public void closeAce() { aceDB.close(); }

   /** 
    * Update the cds table from ACEDB, updates every existing ace_id
    *  and inserting new ace_id; more efficient if cds table is already loaded.
    */
   public void updateFrom(String aceQuery, boolean mRNA) 
   throws SQLException, AceException {
      Result seqSet = aceDB.execQuery(aceQuery);
      PreparedStatement insertStmt= psconn.prepareStatement(
            "INSERT INTO cds(id, ace, prt, length, source, from_mRNA) VALUES(?, ?, ?, ?, ?, ?)");
      insertStmt.setBoolean(6, mRNA);  // once for each load
      PreparedStatement updateStmt= psconn.prepareStatement(
            "UPDATE cds set length=?, source=?, from_mRNA=? where ace=?");
      updateStmt.setBoolean(3, mRNA);

      String aceid = "null", prtid="null";
      String organism="";
      int updateCnt=0, insertCnt=0, skipped=0, length, source;
      while (seqSet.next()) {
         try {
            aceid= seqSet.getString(1);
            if (aceid.length() > ACE_ID_LEN) {
               System.err.println(aceid + "| longer than " + ACE_ID_LEN);
            }
            prtid=seqSet.getString(2);
            length=seqSet.getInt(3);
            source = getOrgid(seqSet.getString(4));

            updateStmt.setString(4, aceid);
            updateStmt.setInt(1, length);
            updateStmt.setInt(2, source);
            if (updateStmt.executeUpdate() == 0) {
               //System.out.println("Cannot update "+aceid+" | "+
               //      length+" | "+source);
               insertCnt++;
               insertStmt.setInt(1, ((Integer)unusedIds.get(unusedIndex++)).intValue());
               insertStmt.setString(2, aceid); 
               if (prtid.equals("")) insertStmt.setNull(3, Types.VARCHAR);
               else insertStmt.setString(3, prtid); 
               insertStmt.setInt(4, length);  
               insertStmt.setInt(5, source); 
               insertStmt.executeUpdate();
            }
            else updateCnt++;
         }
         catch (SQLException se) {
            if (se.toString().indexOf("Cannot insert a duplicate key into unique index cds_ace_key") != -1) {
               System.err.println("cannot insert ace");
               System.exit(1);
               skipped++;
            }
            else {
               System.err.println(insertStmt);
               for (int i=1; i<=seqSet.getColumnCount(); i++) 
                  System.err.print(seqSet.getString(i) + " | ");
               System.out.println("\n");
               System.out.println(se);
               se.printStackTrace();
               System.exit(1);
            }
         }
      }
      System.out.println("Insert cds attempts " + seqSet.getRowCount());
      System.out.println("Total updates " + updateCnt + " total inserts " + insertCnt);
      insertStmt.close();
      updateStmt.close();
   } 

   /** 
    * use this function when the cds table is originally empty,
    * Or all of the entries are mostly new.
    * When called for both CDS and CDS_from_mRNA, usually the second call fails
    * with large databases such as bony, human, etc.
    * I have no solutions yet.  The failer is from the backend.
    */
   public void loadFrom(String aceQuery, boolean mRNA) 
                                 throws SQLException, AceException {
      //System.out.println("Executing the ace query " + aceQuery + " ... ");
      Result seqSet = aceDB.execQuery(aceQuery);
      PreparedStatement insertStmt= psconn.prepareStatement(
            "INSERT INTO cds(id, ace, prt, length, source, from_mRNA) VALUES(?,?,?,?,?,?)");
      insertStmt.setBoolean(6, mRNA);  // once for each load
      String aceid = "", prtid="null";
      String organism="";
      int seqcnt=0;
      int skipped=0;

      while (seqSet.next()) {
         //System.out.println("sequence: " + seqcnt + " skip: " + skipped); // for bugy machines ..
         if (seqcnt%1000 == 0) System.out.print(".");
         try {
            seqcnt++;
            cds_id = nextID();                 // constrols its own id should not get duplicate keys
            insertStmt.setInt(1, cds_id);
            aceid= seqSet.getString(1);
            prtid= seqSet.getString(2);
            insertStmt.setString(2, aceid); 
            if (prtid.equals("")) insertStmt.setNull(3, Types.VARCHAR);
            else insertStmt.setString(3, prtid); 
            insertStmt.setInt(4, seqSet.getInt(3));  

            organism=seqSet.getString(4);  // organism may be empty, this happened with the fugu
            // database, where I loaded a lot of scaffold sequence with giving them the source
            if ( organism.equals("")) {
               String tmp = aceDB.getDBInfo();
               //String tmp1 = aceDB.getDBTitle();
               //System.err.println("info: " + tmp + " title: " + tmp1);
               if (tmp.indexOf("Fugu") != -1) { organism = "Fugu rubripes"; }
               else { System.exit(1); }
            }
            //     insertStmt.setInt(5, getOrgid(seqSet.getString(4)));  // orgname could be null!
             insertStmt.setInt(5, getOrgid(organism));  // orgname could be null!
            insertStmt.executeUpdate();
         }
         catch (SQLException se) {
            if (se.toString().indexOf("Cannot insert a duplicate key into unique index cds_ace_key") != -1) {
               //System.out.println("ace_id already in db: " + aceid);
               skipped++;
            }
            else {
               System.out.println("id=" + cds_id);
               for (int i=1; i<=seqSet.getColumnCount(); i++) 
                  System.err.print(seqSet.getString(i) + " | ");
               System.out.println("\n");
               System.out.println(se);
               se.printStackTrace();
               System.exit(1);
            }
         }
      }
      seqSet.close();
      System.out.println("Insert cds attempts " + seqSet.getRowCount());
      System.out.println("Total skipped " + skipped);
   } 

   /** 
    * create a file for the postgres \Copy command, this can speed up 
    * Database loading by 1000x.  Duging loading also drop index.  This
    * can also greately increase the speed of loading
    */
   public void createFileFrom(String aceQuery, boolean mRNA) 
   throws SQLException, AceException {
      Result seqSet = aceDB.execQuery(aceQuery);
      //PreparedStatement insertStmt= psconn.prepareStatement(
      //      "INSERT INTO cds(id, ace, prt, length, source, from_mRNA) VALUES(?,?,?,?,?,?)");
      //insertStmt.setBoolean(6, mRNA);  // once for each load
      String aceid = "", prtid="null";
      String length;
      String p_cluster = "\\N";
      String n_cluster = "\\N";
      String orgid; // same as source;
      String from_mrna;
      if (mRNA) from_mrna = "t";
      else from_mrna = "f";
      String organism="";
      int seqcnt=0;
      int skipped=0;
      while (seqSet.next()) {
         if (seqcnt%1000 == 0) System.out.print(".");
         try {
            seqcnt++;
            cds_id = nextID();                 // constrols its own id should not get duplicate keys
            aceid= seqSet.getString(1);
            prtid= seqSet.getString(2);
            length = seqSet.getString(3);
            if (prtid.equals("")) prtid = "\\N";

            organism=seqSet.getString(4);  // organism may be empty, this happened with the fugu
            if ( organism.equals("")) {
               String tmp = aceDB.getDBInfo();
               if (tmp.indexOf("Fugu") != -1) { organism = "Fugu rubripes"; }
               else { System.exit(1); }
            }
            orgid = getOrgidString(organism);
            System.out.println(cds_id + " " + aceid + " " + prtid + " " + length + " " + p_cluster + " " + n_cluster + " " + orgid + " " + from_mrna);

         }
         catch (SQLException se) {
               System.out.println("id=" + cds_id);
               for (int i=1; i<=seqSet.getColumnCount(); i++) 
                  System.err.print(seqSet.getString(i) + " | ");
               System.out.println("\n");
               System.out.println(se);
               se.printStackTrace();
               System.exit(1);
         }
      }
      seqSet.close();
      System.out.println("Insert cds attempts " + seqSet.getRowCount());
      System.out.println("Total skipped " + skipped);
   }  

   public void insert() throws SQLException, AceException {
      System.err.println("updating CDS ... ");
      // the problem is that client time out too short for long insertions
      loadFrom(queryCDS, false);  // only when this line is commented out the
                                    // human loading can proceed
      System.err.println("updating CDS from mRNA ... ");
      //loadFrom(querymRNA, true);
   }

   /** 
    * Load all organisms from this acedb to the species table in the
    * relational database if organism already exist then just ignore.
    * So this function can update an existing organism table
    * div provides a division such as rod, vrt
    * depends on postgres sequence to keep track of the id.
    * @param div NCBI division
    */
   public void loadOrganism(String div) throws SQLException, 
                                                    AceException {
      String orgQuery = "select s->species from s in class sequence";
      Result orgSet = aceDB.execQuery(orgQuery);
      PreparedStatement pstmt = 
         psconn.prepareStatement("Insert into species(name, db) values(?, '" + div + "')");
      String thisOrg = "";
      int count=0;
      int skipped =0;
      while (orgSet.next()) {
         try {
            count++;
            thisOrg = orgSet.getString(1);
            if (thisOrg.length() > 80) {  // organism name length max 80 char
               System.err.println(thisOrg + "|| Name longer than 80");
            }
            pstmt.setString(1, thisOrg);
            pstmt.executeUpdate();
         }
         catch (SQLException se) {
            if (se.toString().indexOf("Cannot insert a duplicate key into unique index species_name_key") != -1) {
               
               System.err.println("Species: " +
                     thisOrg + " is already in the postgres DB. " +
                     "We skip this one.");
            
               skipped++;
            }
            else {
               System.err.println("Error in adding species. "+ se);
               System.exit(1);
            }
         }
      }
      pstmt.close();
      System.err.println(count + " Insert organisms attempted\n");
      System.err.println(skipped + " skipped\n");
   }

   /** 
    * does a database query from species table to obtain the id of species
    * from name, orgName must be scientific.
    */
   public int getOrgid(String orgName) throws SQLException {
      orgidStmt.setString(1, orgName);
      ResultSet rs = orgidStmt.executeQuery();
      if (!rs.next()) {
         System.err.println(orgName + " not in database " + pgDB);
         System.exit(1);
      }
      return rs.getInt(1);
   }

   public String getOrgidString(String orgName) throws SQLException {
      orgidStmt.setString(1, orgName);
      ResultSet rs = orgidStmt.executeQuery();
      if (!rs.next()) {
         System.err.println(orgName + " not in database " + pgDB);
         System.exit(1);
      }
      return rs.getString(1);
   }

   /** 
    * To obtain unused cds_id in the cds table, run at 
    * object construction and called by the constructor.
    */
   private Vector getUnusedId() throws SQLException {
      Vector ids = new Vector(130000);
      ResultSet rs = stmt.executeQuery(
            "Select id from cds order by id");
      int cnt=0;
      while (rs.next()) {
         cnt++;
         while (rs.getInt(1) > cnt) {
            ids.add(new Integer(cnt++));
         }
      }
      rs.close();
      System.err.println(ids.size() + " unused ids");
      if (ids.size() == 0) { // no more free unused ids
         rs = stmt.executeQuery("Select max(id) from cds");
         rs.next();
         cds_id=rs.getInt(1);  // always the largest id in the cds table
      }
      return ids;
   }

   private int nextID() {
      if (unusedIndex < unusedIds.size()) 
         return ((Integer)unusedIds.get(unusedIndex++)).intValue();
      else {
         unusedIds.clear();
         return ++cds_id;
      }
   }

   public static void main(String[] args) {
      final String divs[]= { "amp", "bony", "cart", "fugu", "human", "mam", "mouse", "plc", "pri", "rod", "sau", "vrt"};
      int i = 0, startIdx=0;
      boolean org = false, loadDivOnly = false;
      String div=""; //orgName="";
      while (i<args.length) {
         if (args[i].equals("-o"))  org = true;  // load org only
         else if (args[i].equals("-d")) div = args[++i];
         //else if (args[i].equals("-s")) orgName = args[++i];
         else if (args[i].equals("--div")) loadDivOnly = true;
         else if (args[i].equals("-b")) startIdx = Integer.parseInt(args[++i]);  // for reload from failed operation
         i++;
      }

      try {
         LoadCDSFromAce lc = new LoadCDSFromAce();
         if (!div.equals("")) {  // loading from specific database
            lc.connect( (String)acename2host.get(div), ((Integer)acename2port.get(div)).intValue());
            if (org) lc.loadOrganism(div);
            else lc.insert();
            lc.closeAce();
         }
         else {
            // load from all databases
            for (i=startIdx; i<divs.length; i++) {
               System.out.println("\nWorking on " + divs[i]);
               lc.connect((String)acename2host.get(divs[i]), ((Integer)acename2port.get(divs[i])).intValue());
               if (org) lc.loadOrganism(divs[i]);
               else lc.insert();
               lc.closeAce();
            }
            if (loadDivOnly) { 
               lc.close(); 
               return;
            }
         }
         lc.close();
      }
      catch (AceException ae) {
         System.err.println("Ace error " + ae);
         ae.printStackTrace();
      }
      catch (SQLException sqle) {
         System.err.println(sqle);
         sqle.printStackTrace();
      }
      catch (ClassNotFoundException ce) {
         System.out.println(ce);
         ce.printStackTrace();
         System.exit(1);
      }
   }
}

let me know you need more.  It crashed in the lexer part I am not an expert on lex.  If it was pure C++ or C I could debug it for you.
Comment 5 albert 2016-04-09 11:59:16 UTC
I've just pushed a proposed patch to github (pull request 467)
Comment 6 albert 2016-04-10 07:54:16 UTC
Code has ben integrated in the main branch on github.
Comment 7 Dimitri van Heesch 2016-09-05 13:45:35 UTC
This bug was previously marked ASSIGNED, which means it should be fixed in
doxygen version 1.8.12. Please verify if this is indeed the case. Reopen the
bug if you think it is not fixed and please include any additional information 
that you think can be relevant (preferably in the form of a self-contained example).