[jcifs] Unexpected fragment length exception when retreiving ACLs/SIDs on a file that has explicit permission granted to a deleted user

[Colin Dean]

Hello,

I’m encountering an unexpected exception while retrieving SIDs/ACLs for a
file served by Windows Server 2012 Standard Edition. We’re running on
Windows in this instance, if that matters.

Caused by: java.io.IOException: Unexpected fragment length: 28757
           at jcifs.dcerpc.DcerpcPipeHandle.doReceiveFragment
(DcerpcPipeHandle.java:96)
           at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:220)
at jcifs.smb.SID.resolveSids(SID.java:94)
           at com.vivisimo.connector.smb.SMBConnector.resolveSids
(SMBConnector.java:629) ... 10 more

This is approximately where our code hands off to jcifs to resolve the
SIDs:

SID[] sidsToResolveArray = sidsToResolve.toArray(new SID[0]);
try {
LsaPolicyHandle policyHandle = null;
try {

policyHandle = handle.openPolicyHandle(file);
SID.resolveSids(handle.getHandle(), policyHandle,
sidsToResolveArray);

} finally {
if (policyHandle != null) {
LOGGER.debug("Closing Policy handle for [" + file +
"]");
policyHandle.close();
}
policyHandle = null;
}
} catch (IOException e) {
throw new ConnectorException(e);
}

It dies shortly after we call SID.resolveSids() there in the middle of the
inner try.

We have a few theories:

1. This filesystem has a lot of files with Japanese characters in the
filename. We considered that perhaps jCIFS or our software using it lacks
support for Japanese characters, but we set up a testing environment with a
bunch of files with Japanese filenames and it works for us. We’re pretty
certain that our configuration mirrors that of the erring environment, but
there might be subtle differences were are not able to replicate (e.g. real
data instead of made up test data).

2. All of the files that are failing (we’re crawling a filesystem) share
one trait: they have what appears in the Windows properties a permitted
user that shows as “unknown user”, which may be an artifact remaining after
the user was removed but its explicit permissions remained. Processing on a
directory that has no “unknown users” yields no failures.

This leads me to believe that jCIFS is failing on unknown users. However,
looking through the source, I don’t see an obvious way to detect this.

The obvious workaround appears to be “clean your unknown users” but I think
this could be detectable and loggable during our processing of the share.

I’d appreciate input on this.




Colin Dean
Advisory Software Engineer &
Tech Lead, Watson Explorer
Connectivity
email: ***@us.ibm.com |
tel: [phone]
Find me on: LinkedIn:
http://linkedin.com/in/colindean
Github:
http://github.com/colindean
Launchpad:
http://launchpad.net/~colindean


IBM Watson Group Watson on Facebook: http://www.facebook.com/ibmwatson
Watson on Twitter: https://twitter.com/@ibmwatson
Watson on YouTube:
http://www.youtube.com/user/IBMWatsonSolutions/videos

[Colin Dean]

Although the changelog does not appear at glance to have addressed a
problem that could cause this error, I should note that we're using jcifs
1.3.15 in this particular case. We're in the process of deploying 1.3.18.

From: Colin Dean/Pittsburgh/IBM
Date: 04/27/2015 15:09
Subject: Unexpected fragment length exception when retreiving ACLs/
SIDs on a file that has explicit permission granted to a deleted user
Hello,
I’m encountering an unexpected exception while retrieving SIDs/ACLs
for a file served by Windows Server 2012 Standard Edition. We’re
running on Windows in this instance, if that matters.
Caused by: java.io.IOException: Unexpected fragment length: 28757
           at jcifs.dcerpc.DcerpcPipeHandle.doReceiveFragment
(DcerpcPipeHandle.java:96)
           at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:220)
at jcifs.smb.SID.resolveSids(SID.java:94)
           at com.vivisimo.connector.smb.SMBConnector.resolveSids
(SMBConnector.java:629) ... 10 more
This is approximately where our code hands off to jcifs to resolve the
SID[] sidsToResolveArray = sidsToResolve.toArray(new SID[0]);
try {
LsaPolicyHandle policyHandle = null;
try {
policyHandle = handle.openPolicyHandle(file);
SID.resolveSids(handle.getHandle(),
policyHandle, sidsToResolveArray);
} finally {
if (policyHandle != null) {
LOGGER.debug("Closing Policy handle for [" +
file + "]");
policyHandle.close();
}
policyHandle = null;
}
} catch (IOException e) {
throw new ConnectorException(e);
}
It dies shortly after we call SID.resolveSids() there in the middle
of the inner try.
1. This filesystem has a lot of files with Japanese characters in
the filename. We considered that perhaps jCIFS or our software using
it lacks support for Japanese characters, but we set up a testing
environment with a bunch of files with Japanese filenames and it
works for us. We’re pretty certain that our configuration mirrors
that of the erring environment, but there might be subtle
differences were are not able to replicate (e.g. real data instead
of made up test data).
2. All of the files that are failing (we’re crawling a filesystem)
share one trait: they have what appears in the Windows properties a
permitted user that shows as “unknown user”, which may be an
artifact remaining after the user was removed but its explicit
permissions remained. Processing on a directory that has no “unknown
users” yields no failures.
This leads me to believe that jCIFS is failing on unknown users.
However, looking through the source, I don’t see an obvious way to
detect this.
The obvious workaround appears to be “clean your unknown users” but
I think this could be detectable and loggable during our processing
of the share.
I’d appreciate input on this.
Colin Dean
Advisory Software Engineer & Tech Lead, Watson Explorer Connectivity
Find me on: [image removed]  [image removed]  [image removed]
[image removed]
[image removed] [image removed] [image removed]
[image removed]

[Michael B Allen]

Hi Colin,

This is a buffering error at the DCERPC layer. During the DCERPC bind, the
client submits max_xmit and max_recv values that negotiate the largest
buffer size that the client will transmit / receive. This value is
hardcoded in JCIFS at 4280. This is observed Windows behavior and I don't
recall ever seeing an issue with buffering like you're experiencing.

But apparently the server is returning a buffer that is much larger
(28757). At first glance, this might look like a decoding error. Meaning
the buffer begin decoded as a DCERPC message is actually total garbage. But
I don't think this is the case because there is a DCERPC header sanity
check just before the frag size check. So it looks like a legit DCERPC
response. It's just that the fragment size is way to large for JCIFS to
handle.

I do recall that the max_xmit and max_recv values are actually suggested
sizes and that the server can actually return more data. But I could be
mis-remembering that.

The Japanese characters could be involved but I somewhat doubt that it has
anything to do with the ability to interpret the actual characters. It's
more likely just a buffer size issue. If the SID names are Japanese
strings, their encoding could be unexpectedly large.

Just as a sanity check, try examples/SidLookup.java from the JCIFS package
with one of the offending SIDs. What gets printed on the console might not
be legible unless your on a Japanese workstation but it might still be
enlightening.

I would need to see a packet capture to diagnose this further (do not post
captures to the mailing list! - send them to me directly). If you can get
good network packet captures of a Windows client successfully resolving the
SIDs and then another of JCIFS failing to resolve those same SIDs, then I
would be happy to at least look at them. But to get good captures might be
tricky. You might need to write a JCIFS program that takes a list of SIDs
and calls resolveSids. And for Windows you might need to reboot the
workstation, start the capture and then look at the ACL on the file using
the file browser.

If you look at the ACL of the file in the file browser, are the group /
account names displayed correctly / successfully? Are there a lot?

Unfortunately if it turns out that JCIFS is not correctly handling some
kind of dynamic buffering, that could be a difficult fix. But you might be
able to workaround by catching that particular exception and then
recursively re-trying resolveSids with fewer SIDs (meaning divide the
operation into 2 SID arrays and if it still fails, recur on each array with
some depth limit). But I'm just thinking out loud. I would need to see
captures to make a proper diagnosis.

Mike

PS: Do NOT send network packet captures to the mailing list. Send them only
to me directly.

Post by Colin Dean
Hello,
I’m encountering an unexpected exception while retrieving SIDs/ACLs for a
file served by Windows Server 2012 Standard Edition. We’re running on
Windows in this instance, if that matters.
*Caused by: java.io.IOException: Unexpected fragment length: 28757*
* at
jcifs.dcerpc.DcerpcPipeHandle.doReceiveFragment(DcerpcPipeHandle.java:96)*
* at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:220) *
* at jcifs.smb.SID.resolveSids(SID.java:94)*
* at
com.vivisimo.connector.smb.SMBConnector.resolveSids(SMBConnector.java:629)
... 10 more*
This is approximately where our code hands off to jcifs to resolve the
SID[] sidsToResolveArray = sidsToResolve.toArray(new SID[0]);
try {
LsaPolicyHandle policyHandle = null;
try {
policyHandle = handle.openPolicyHandle(file);
SID.resolveSids(handle.getHandle(), policyHandle, sidsToResolveArray);
} finally {
if (policyHandle != null) {
LOGGER.debug("Closing Policy handle for [" + file + "]");
policyHandle.close();
}
policyHandle = null;
}
} catch (IOException e) {
throw new ConnectorException(e);
}
It dies shortly after we call SID.resolveSids() there in the middle of the
inner try.
1. This filesystem has a lot of files with Japanese characters in the
filename. We considered that perhaps jCIFS or our software using it lacks
support for Japanese characters, but we set up a testing environment with a
bunch of files with Japanese filenames and it works for us. We’re pretty
certain that our configuration mirrors that of the erring environment, but
there might be subtle differences were are not able to replicate (e.g. real
data instead of made up test data).
2. All of the files that are failing (we’re crawling a filesystem) share
one trait: they have what appears in the Windows properties a permitted
user that shows as “unknown user”, which may be an artifact remaining after
the user was removed but its explicit permissions remained. Processing on a
directory that has no “unknown users” yields no failures.
This leads me to believe that jCIFS is failing on unknown users. However,
looking through the source, I don’t see an obvious way to detect this.
The obvious workaround appears to be “clean your unknown users” but I
think this could be detectable and loggable during our processing of the
share.
I’d appreciate input on this.
------------------------------
*Colin DeanAdvisory Software Engineer & Tech Lead, Watson Explorer
LinkedIn: http://linkedin.com/in/colindean]
Launchpad: http://launchpad.net/~colindean]
<http://launchpad.net/~colindean>
http://www.youtube.com/user/IBMWatsonSolutions/videos]
<http://www.youtube.com/user/IBMWatsonSolutions/videos>
------------------------------

[Colin Dean]

Thanks for the recommendations. I'll work on collecting those captures and
give the example a try. In the group/account name, there are several
"unknown account" entries. There are 127 of them of ~330 entries total.

This will take some time. There's a time delay between my time and the
folks I'm working with, plus I'm out on vacation until 5/3.




Colin Dean
Advisory Software Engineer &
Tech Lead, Watson Explorer
Connectivity
email: ***@us.ibm.com |
tel: [phone]
Find me on: LinkedIn:
http://linkedin.com/in/colindean
Github:
http://github.com/colindean
Launchpad:
http://launchpad.net/~colindean


IBM Watson Group Watson on Facebook: http://www.facebook.com/ibmwatson
Watson on Twitter: https://twitter.com/@ibmwatson
Watson on YouTube:
http://www.youtube.com/user/IBMWatsonSolutions/videos










From: Michael B Allen <***@gmail.com>
To: Colin Dean/Pittsburgh/***@IBMUS
Cc: "***@lists.samba.org" <***@lists.samba.org>
Date: 04/28/2015 11:35
Subject: Re: [jcifs] Unexpected fragment length exception when
retreiving ACLs/SIDs on a file that has explicit permission
granted to a deleted user



Hi Colin,

This is a buffering error at the DCERPC layer. During the DCERPC bind, the
client submits max_xmit and max_recv values that negotiate the largest
buffer size that the client will transmit / receive. This value is
hardcoded in JCIFS at 4280. This is observed Windows behavior and I don't
recall ever seeing an issue with buffering like you're experiencing.

But apparently the server is returning a buffer that is much larger
(28757). At first glance, this might look like a decoding error. Meaning
the buffer begin decoded as a DCERPC message is actually total garbage. But
I don't think this is the case because there is a DCERPC header sanity
check just before the frag size check. So it looks like a legit DCERPC
response. It's just that the fragment size is way to large for JCIFS to
handle.

I do recall that the max_xmit and max_recv values are actually suggested
sizes and that the server can actually return more data. But I could be
mis-remembering that.

The Japanese characters could be involved but I somewhat doubt that it has
anything to do with the ability to interpret the actual characters. It's
more likely just a buffer size issue. If the SID names are Japanese
strings, their encoding could be unexpectedly large.

Just as a sanity check, try examples/SidLookup.java from the JCIFS package
with one of the offending SIDs. What gets printed on the console might not
be legible unless your on a Japanese workstation but it might still be
enlightening.

I would need to see a packet capture to diagnose this further (do not post
captures to the mailing list! - send them to me directly). If you can get
good network packet captures of a Windows client successfully resolving the
SIDs and then another of JCIFS failing to resolve those same SIDs, then I
would be happy to at least look at them. But to get good captures might be
tricky. You might need to write a JCIFS program that takes a list of SIDs
and calls resolveSids. And for Windows you might need to reboot the
workstation, start the capture and then look at the ACL on the file using
the file browser.

If you look at the ACL of the file in the file browser, are the group /
account names displayed correctly / successfully? Are there a lot?

Unfortunately if it turns out that JCIFS is not correctly handling some
kind of dynamic buffering, that could be a difficult fix. But you might be
able to workaround by catching that particular exception and then
recursively re-trying resolveSids with fewer SIDs (meaning divide the
operation into 2 SID arrays and if it still fails, recur on each array with
some depth limit). But I'm just thinking out loud. I would need to see
captures to make a proper diagnosis.

Mike

PS: Do NOT send network packet captures to the mailing list. Send them only
to me directly.

On Mon, Apr 27, 2015 at 3:09 PM, Colin Dean <***@us.ibm.com> wrote:
Hello,

I’m encountering an unexpected exception while retrieving SIDs/ACLs for a
file served by Windows Server 2012 Standard Edition. We’re running on
Windows in this instance, if that matters.

Caused by: java.io.IOException: Unexpected fragment length: 28757
           at jcifs.dcerpc.DcerpcPipeHandle.doReceiveFragment
(DcerpcPipeHandle.java:96)
           at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:220)
at jcifs.smb.SID.resolveSids(SID.java:94)
           at com.vivisimo.connector.smb.SMBConnector.resolveSids
(SMBConnector.java:629) ... 10 more

This is approximately where our code hands off to jcifs to resolve the
SIDs:

SID[] sidsToResolveArray = sidsToResolve.toArray(new SID[0]);
try {
LsaPolicyHandle policyHandle = null;
try {

policyHandle = handle.openPolicyHandle(file);
SID.resolveSids(handle.getHandle(), policyHandle, sidsToResolveArray);

} finally {
if (policyHandle != null) {
LOGGER.debug("Closing Policy handle for [" + file + "]");
policyHandle.close();
}
policyHandle = null;
}
} catch (IOException e) {
throw new ConnectorException(e);
}

It dies shortly after we call SID.resolveSids() there in the middle of
the inner try.

We have a few theories:

1. This filesystem has a lot of files with Japanese characters in the
filename. We considered that perhaps jCIFS or our software using it lacks
support for Japanese characters, but we set up a testing environment with
a bunch of files with Japanese filenames and it works for us. We’re
pretty certain that our configuration mirrors that of the erring
environment, but there might be subtle differences were are not able to
replicate (e.g. real data instead of made up test data).

2. All of the files that are failing (we’re crawling a filesystem) share
one trait: they have what appears in the Windows properties a permitted
user that shows as “unknown user”, which may be an artifact remaining
after the user was removed but its explicit permissions remained.
Processing on a directory that has no “unknown users” yields no failures.

This leads me to believe that jCIFS is failing on unknown users. However,
looking through the source, I don’t see an obvious way to detect this.

The obvious workaround appears to be “clean your unknown users” but I
think this could be detectable and loggable during our processing of the
share.

I’d appreciate input on this.







Colin Dean
Advisory Software Engineer &
Tech Lead, Watson Explorer
Connectivity
email: ***@us.ibm.com |
tel: [phone]
Find me on: LinkedIn:
http://linkedin.com/in/colindean
Github:
http://github.com/colindean
Launchpad:
http://launchpad.net/~colindean


IBM Watson Group Watson on Facebook: http://www.facebook.com/ibmwatson
Watson on Twitter: https://twitter.com/@ibmwatson
Watson on YouTube:
http://www.youtube.com/user/IBMWatsonSolutions/videos