[issue29562] test_getgroups of test_posix fails (on OS X 10.10)

[Jim DeLaHunt]

Jim DeLaHunt added the comment:

Some diagnosis.

Group `com.apple.sharepoint.group.1` appears to be related to a certain kind of file sharing, but I don't have hard evidence. 

Its only member was a test user I created as part of screen sharing with Apple Support. 
```
% dscacheutil -q group -a name com.apple.sharepoint.group.1
name: com.apple.sharepoint.group.1
password: *
gid: 701
users: testuser
```

I removed File Sharing for this user's home directory.

1. Open System Preferences... Sharing. 
2. Click on "File Sharing", which is checked. In the right pane, a list of shared folders appears.
3. Click on the entry "Testuser Public Folder" in the Shared Folders list.
4. Click on the "-" button below the Shared Folders list. The "Testuser Public Folder" entry disappears.

Having done that, the group `com.apple.sharepoint.group.1` no longer appeared.

```
% dscacheutil -q group -a name com.apple.sharepoint.group.1
%
```

Interestingly, `test_getgroups` still failed, and still had a discrepancy of two groups from the output of `id -G`.

```
% ./python.exe -m unittest -v test.test_posix.PosixTester.test_getgroups
test_getgroups (test.test_posix.PosixTester) ... FAIL

======================================================================
FAIL: test_getgroups (test.test_posix.PosixTester)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jdlh/workspace/cpython/Lib/test/test_posix.py", line 841, in test_getgroups
    self.assertEqual(len(symdiff), 0, msg)
AssertionError: 2 != 0 : id -G and posix.groups() should have zero difference.
Groups in id -G but not posix.groups(): [(395, 'com.apple.access_ftp'), (398, 'com.apple.access_screensharing')]
Groups in posix.groups() but not id -G: []
(Effective GID (20) was disregarded.)

----------------------------------------------------------------------
Ran 1 test in 0.013s

FAILED (failures=1)
```

Earlier, group `com.apple.access_ftp` was not part of the difference. Now it is. The output of `id -G` didn't change. The implementation of `posix.getgroups()` didn't change. It calls getgroups (2), I believe: https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L6078-L6103

That makes me think that the behaviour of getgroups (2) in Mac OS is behaving differently than we expect. 

`man 2 getgroups` gives documentation. (I can't find this page at an apple URL, but http://www.manpagez.com/man/2/getgroups/ seems to have the same content.) It says, 

>>> "To provide compatibility with applications that use getgroups() in environments where users may be in more than {NGROUPS_MAX} groups, a variant of getgroups(), obtained when compiling with either the macros _DARWIN_UNLIMITED_GETGROUPS or _DARWIN_C_SOURCE defined, can be used that is not limited to {NGROUPS_MAX} groups.  However, this variant only returns the user's default group access list and not the group list modified by a call to setgroups(2) (either in the current process or an ancestor process).  Use of setgroups(2) is highly discouraged, and there is no foolproof way to determine if it has been previously called."

I don't know how to determine if my copy of Mac OS X 10.10 was complied with either of these two macros. 

On my system, I chased NGROUPS_MAX down to /usr/include/sys/syslimits.h:84, where it is set to 16. That is more than the number of groups `id -G` is reporting, so I don't see how that is relevant.

```% id -G
20 507 12 61 80 98 399 33 100 204 395 398
```

This is 12 groups, whereas before it was 13 groups (see my message from 2017-02-15 02:03). This is unsurprising.  However, the number of groups returned by posix.getgroups() has also shrunk by 1:

```% ./python.exe -c 'import grp,os; g={i: (n, p, i, mem) for (n, p, i, mem) in grp.getgrall()}; print(sorted([(i, g[i][0]) for i in os.getgroups()]) )'
[(12, 'everyone'), (20, 'staff'), (33, '_appstore'), (61, 'localaccounts'), (80, 'admin'), (98, '_lpadmin'), (100, '_lpoperator'), (204, '_developer'), (399, 'com.apple.access_ssh'), (507, 'xampp')]
```

Notice that group (395, 'com.apple.access_ftp') is no longer being returned by os.getgroups().  This is as a consequence of a different group being deleted.

The test_getgroups comment asserts: "# 'id -G' and 'os.getgroups()' should return the same groups, ignoring order, duplicates, and the effective gid." https://github.com/python/cpython/blob/master/Lib/test/test_posix.py#L819-L820

I'm getting skeptical about that claim. Does Mac OS X actually guarantee that 'id -G' and 'getgroups(2)' return the same groups?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29562>
_______________________________________