After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 646268 - "Import MT940 file" does not work - "missing bank reference"
"Import MT940 file" does not work - "missing bank reference"
Status: RESOLVED FIXED
Product: GnuCash
Classification: Other
Component: Import - AqBanking
2.4.x
Other All
: Normal normal
: ---
Assigned To: Christian Stimming
Christian Stimming
: 647302 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2011-03-30 20:28 UTC by Daniel Schmid
Modified: 2018-06-29 22:55 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Backtrace of mt940 hang (12.52 KB, text/plain)
2011-04-18 20:43 UTC, John Ralls
Details

Description Daniel Schmid 2011-03-30 20:28:06 UTC
I try to import a mt940 file for a back account. when inporting the same file on Windows PC, it works. When importing on OsX version, there is no window coming up, the import command does nothing...
Comment 1 Christian Stimming 2011-03-31 08:53:00 UTC
What error message is in the trace file http://wiki.gnucash.org/wiki/Tracefile ?
Comment 2 Christian Stimming 2011-03-31 08:53:53 UTC
(@John: There might be some aqbanking or gwenhywfar plugin missing, related to mt940 files...)
Comment 3 John Ralls 2011-03-31 13:32:56 UTC
Would it be installed somewhere other than lib/gwenhywfar or lib/aqbanking? I'm pretty sure I'm getting all of the plugins from those directories.
Comment 4 Christian Stimming 2011-03-31 13:53:36 UTC
There are two shared libraries involved: lib/gwenhywfar/plugins/xx/dbio/swift.{xml,so} and lib/aqbanking/plugins/xx/imexporters/swift.{xml,so}
Comment 5 Christian Stimming 2011-03-31 14:06:29 UTC
... and there is one more file, the "profile" (better: "format spec file") in share/aqbanking/imexporters/swift/profiles/swiftmt940.conf. Sigh.
Comment 6 John Ralls 2011-03-31 15:15:00 UTC
All present in the bundle. Is there a test file anywhere that I can use?
Comment 7 Christian Stimming 2011-03-31 16:35:19 UTC
In gnucash: doc/examples/downloaded.mt940
Comment 8 John Ralls 2011-03-31 20:03:20 UTC
When I try from the bundle (with aqbanking-5.0.2 and gwen-4.0.4) I get the same results as the OP, with the following unhelpful error messages:

* 13:01:48  CRIT <aqbanking> swift.c:  106: Error importing data (-57)
* 13:01:48  WARN <gnc.import.aqbanking> gnc_file_aqbanking_import: Error on import

Worse, when I try to run it from the command line in the original installation directory, this time using freshly updated svn builds of each, it hangs in this loop at GWEN_SyncIo_File_Read (sio=0x349d4d0, buffer=0x1cf01bec "", size=256) at syncio_file.c:312

  do {
    rv=read(xio->fd, buffer, size);
  } while (rv==-1 && errno==EINTR);



with the following values:

$6 = {
  path = 0x3498590 "/Volumes/RAID1/Gnucash-Build/Gnucash-svn/src/gnucash-git/doc/examples/downloaded.mt940", 
  creationMode = GWEN_SyncIo_File_CreationMode_OpenExisting, 
  fd = 0
}
(gdb) p *sio
$7 = {
  INHERIT__list = 0x3497ed0, 
  _list1_element = 0x1cf007a0, 
  refCount = 1, 
  baseIo = 0x0, 
  typeName = 0x1cf016b0 "file", 
  flags = 0, 
  status = GWEN_SyncIo_Status_Unconnected, 
  connectFn = 0x19f82835 <GWEN_SyncIo_File_Connect>, 
  disconnectFn = 0x19f82d29 <GWEN_SyncIo_File_Disconnect>, 
  flushFn = 0, 
  readFn = 0x19f82f9b <GWEN_SyncIo_File_Read>, 
  writeFn = 0x19f831e2 <GWEN_SyncIo_File_Write>
}

Heh. Ain't getting input on fd0 (aka stdin). Martin's code is too impenetrable for me to easily figure out why it's trying to read stdin instead of the file path, and I have too much other stuff to do. Sorry.
Comment 9 Christian Stimming 2011-04-18 12:01:57 UTC
Can you give me the full backtrace for this hang? I'd insert some sanity checking code (and/or maybe debug output) at the point where the GWEN_SYNCIO_FILE is created but the filedescriptor is still zero.
Comment 10 John Ralls 2011-04-18 20:43:30 UTC
Created attachment 186232 [details]
Backtrace of mt940 hang

Here ya go!
Comment 11 Christian Stimming 2011-04-19 07:02:34 UTC
The GWEN_SYNCIO is created on the gnucash side in gnc-file-aqb-import.c line 179. Seems we need to add more sanity checks there...
Comment 12 Christian Stimming 2011-04-19 08:07:22 UTC
In gnucash r20565 I have (hopefully) fixed the initialization on the gnucash side. Does this now work?

If my fix is correct, it means nobody has been able to import a file using aqbanking5 so far, only with aqbanking4.
Comment 13 Christian Stimming 2011-04-19 10:03:09 UTC
*** Bug 647302 has been marked as a duplicate of this bug. ***
Comment 14 John Ralls 2011-04-19 19:17:30 UTC
Well, good news and bad news.

After fixing a minor oops (committed in r20569), it doesn't hang anymore. The bad news is that it can't read the file, producing a blank import dialog box and the following errors to the terminal:

LibOFX INFO: libofx_proc_file(): File format not specified, autodecting...
(Above message occured on Line 4294967295, Column 4294967295)
LibOFX ERROR: libofx_detect_file_type(): Failed to identify input file format
(Above message occured on Line 4294967295, Column 4294967295)
LibOFX INFO: libofx_proc_file(): Detected file format: UNKNOWN (File format couldn't be sucessfully identified)
(Above message occured on Line 4294967295, Column 4294967295)
LibOFX ERROR: libofx_proc_file(): Detected file format not yet supported ou couldn't detect file format; aborting.
(Above message occured on Line 4294967295, Column 4294967295)

I didn't see anything useful in the trace file.
Comment 15 Christian Stimming 2011-04-19 20:19:26 UTC
You tried the "mt940" import but got error messages from libofx?!?
Comment 16 John Ralls 2011-04-19 21:36:32 UTC
Oops, sorry, I tried to import it as an OFX.

Try again. It doesn't hang, and puts up a Gwen window with the messages:
14:35:06 SWIFT: Missing bank reference
14:35:06 Operation finished, you can now close this window.

There's no evidence that it imports anything.
Comment 17 Christian Stimming 2011-04-20 18:30:14 UTC
I get the same error on Linux, so it's not a Mac specific thing.
Comment 18 Christian Stimming 2011-04-20 18:35:53 UTC
The same error can be reproduced by the following aqbanking-cli command:

  aqbanking-cli import --importer=swift --profile=mt940 -f downloaded.mt940

where the content of "downloaded.mt940" is this:

:20:STARTUMS  
:25:80007777/2602272001
:28C:0
:60F:C060228EUR0,00
:61:0604280228D12,56N020NONREF//
:86:020?00ÜBERWEISUNG?100?20TESTING GNUCASH FOR HBCI?21AND MT940
?3080007777?312602272003?32GNUCASH?34000
:62F:D060228EUR12,56
-
Comment 19 Christian Stimming 2011-04-20 18:54:52 UTC
Fixed in aqbanking r2180, to become aqbanking-5.0.6.
Comment 20 John Ralls 2011-04-28 17:21:34 UTC
Hmm. I have a build problem. When I run the aqbanking-cli line above I get the following error:

$ manjusri:ash-svn/src/gnucash-git> aqbanking-cli import --importer=swift --profile=mt940 -f doc/examples/downloaded.mt940 
3:2011/04/28 10-08-06:aqbanking(10870):banking_imex.c:  745: Profile "mt940" for importer "swift" not found
3:2011/04/28 10-08-06:aqbanking-cli(10870):import.c:  180: Error reading file: -51

Is there an AQBanking environment variable that I need to set to tell it to look in $PREFIX/share/aqbanking/imexporters/swift/profiles (which is where I find swiftmt940.conf)?
Comment 21 a2235158 2011-05-01 16:50:27 UTC
I copied the swiftmt940.conf to my local profile directory
$HOME/.aqbanking/imexporters/swift/profiles/


Then, I set aqbanking the debuglevel for AQBanking to debug:

export AQBANKING_LOGLEVEL=debug

When you do this, and enter the above command I get the following output:


6:2011/05/01 18-47-26:aqbanking(3391):banking_init.c:   63: AqBanking v5.0.4.0stable (compiled at 20110501173634): initialising
6:2011/05/01 18-47-26:aqbanking(3391):banking_init.c:  180: Registering bankinfo plugin manager
6:2011/05/01 18-47-26:aqbanking(3391):banking_init.c:  212: Registering provider plugin manager
6:2011/05/01 18-47-26:aqbanking(3391):banking_init.c:  243: Registering imexporters plugin manager
6:2011/05/01 18-47-26:aqbanking(3391):banking_cfg.c:  561: No users
6:2011/05/01 18-47-26:aqbanking(3391):provider.c:  454: No update function set
6:2011/05/01 18-47-26:aqbanking(3391):provider.c:  454: No update function set
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "/usr/share/aqbanking/imexporters/swift/profiles/swiftmt942.conf" contains profile "SWIFT-MT942"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "/usr/share/aqbanking/imexporters/swift/profiles/amro940.conf" contains profile "AMRO-MT940"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "/usr/share/aqbanking/imexporters/swift/profiles/amro942.conf" contains profile "AMRO-MT942"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "/usr/share/aqbanking/imexporters/swift/profiles/swiftmt940.conf" contains profile "SWIFT-MT940"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "$HOME/.aqbanking/imexporters/swift/profiles/amro940.conf" contains profile "AMRO-MT940"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "$HOME/.aqbanking/imexporters/swift/profiles/swiftmt940.conf" contains profile "SWIFT-MT940"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "$HOME/.aqbanking/imexporters/swift/profiles/swiftmt942.conf" contains profile "SWIFT-MT942"
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  371: File "$HOME/.aqbanking/imexporters/swift/profiles/amro942.conf" contains profile "AMRO-MT942"
3:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  745: Profile "mt940" for importer "swift" not found
6:2011/05/01 18-47-26:aqbanking(3391):banking_imex.c:  797: here (-51)
3:2011/05/01 18-47-26:aqbanking-cli(3391):import.c:  180: Error reading file: -51

so this clearly seems to indicate, that it is an aqbanking bug, as aqbanking is initially finding the profile.
Comment 22 John Ralls 2011-05-16 20:29:00 UTC
I just confirmed that this is true in Linux as well for both aqbanking-5.0.2 and a subversion checkout today.
Comment 23 John Ralls 2011-05-16 21:25:39 UTC
Aha! I broke the code: The correct command line is 

aqbanking-cli import --importer=swift --profile=SWIFT-MT940 -f downloaded.mt940


Tested satisfactorily from gnucash too. Now if we can get Martin to spin a new tarball, I can put it in the next release and close this.
Comment 24 Christian Stimming 2011-05-21 20:15:50 UTC
There is now an aqbanking-5.0.6 version released yesterday.
Comment 25 John Ralls 2011-05-21 22:02:52 UTC
Yup. I've already copied it to my github account (Martin's site doesn't support automated downloads) and set up the moduleset. I've been working on fixing up the windows menu on OSX so that it doesn't spew into the trace file; I think I've got another day of work on that. We can do a release after that, whenever you and Phil are ready.
Comment 26 Christian Stimming 2011-11-02 13:54:41 UTC
Isn't that fixed now?
Comment 27 John Ralls 2011-11-03 00:27:09 UTC
Yes, in fact 2.4.8 has 5.0.10. Thanks for reminding me about the bug.
Comment 28 John Ralls 2017-09-24 22:41:36 UTC
Reassign version to 2.4.x so that individual 2.4 versions can be retired.
Comment 29 John Ralls 2018-06-29 22:55:55 UTC
GnuCash bug tracking has moved to a new Bugzilla host. This bug has been copied to https://bugs.gnucash.org/show_bug.cgi?id=646268. Please update any external references or bookmarks.