more Java scons3 issues...

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

more Java scons3 issues...

Mats Wichmann

I'm running into this:

Code snippet:

jdk_env.Java(target='classes', source=['src/main/java'])
example_jar = jdk_env.Jar(target='simpleclientserver.jar',
                          source=['classes', 'MANIFEST.MF'],
                          JARCHDIR='$SOURCE')

When run on 2.5.1, as expected (since the above stanza was adjusted
until it worked - 'classes' is in the source instead of the built
classes due to the Java() problem that it can't always reliably detect
what javac will emit, the case described in
http://scons.tigris.org/issues/show_bug.cgi?id=2547 being in play here)

jar cfm
out/linux/x86_64/debug/java/examples-java/simpleclientserver/simpleclientserver.jar
java/examples-java/simpleclientserver/MANIFEST.MF -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes .


When run on 3.0.1:

jar cfm
out/linux/x86_64/debug/java/examples-java/simpleclientserver/simpleclientserver.jar
java/examples-java/simpleclientserver/MANIFEST.MF -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes .
-C out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org/iotivity -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org/iotivity/base -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org/iotivity/base/examples
java.util.zip.ZipException: duplicate entry: org/


After some experimentation, it looks like this problem is taken care of
if I drop in the latest Tool/jar.py from github:

jar cf
out/linux/x86_64/debug/java/examples-java/simpleclientserver/simpleclientserver.jar
-C out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes .


but... now the manifest isn't being included.

ideas?
_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Bill Deegan-2
Mats,

I see you're filing a bug for this.
If you check the bug filing policy we ask that you wait for someone to approve filing a new bug to prevent duplicates, etc from filling up the bug tracker.

Have you tried the lastest changes in the master branch?
There are some new Java fixes in there since 3.0.1..

-Bill

On Fri, Dec 15, 2017 at 12:55 PM, Mats Wichmann <[hidden email]> wrote:

I'm running into this:

Code snippet:

jdk_env.Java(target='classes', source=['src/main/java'])
example_jar = jdk_env.Jar(target='simpleclientserver.jar',
                          source=['classes', 'MANIFEST.MF'],
                          JARCHDIR='$SOURCE')

When run on 2.5.1, as expected (since the above stanza was adjusted
until it worked - 'classes' is in the source instead of the built
classes due to the Java() problem that it can't always reliably detect
what javac will emit, the case described in
http://scons.tigris.org/issues/show_bug.cgi?id=2547 being in play here)

jar cfm
out/linux/x86_64/debug/java/examples-java/simpleclientserver/simpleclientserver.jar
java/examples-java/simpleclientserver/MANIFEST.MF -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes .


When run on 3.0.1:

jar cfm
out/linux/x86_64/debug/java/examples-java/simpleclientserver/simpleclientserver.jar
java/examples-java/simpleclientserver/MANIFEST.MF -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes .
-C out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org/iotivity -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org/iotivity/base -C
out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes
org/iotivity/base/examples
java.util.zip.ZipException: duplicate entry: org/


After some experimentation, it looks like this problem is taken care of
if I drop in the latest Tool/jar.py from github:

jar cf
out/linux/x86_64/debug/java/examples-java/simpleclientserver/simpleclientserver.jar
-C out/linux/x86_64/debug/java/examples-java/simpleclientserver/classes .


but... now the manifest isn't being included.

ideas?
_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users


_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Mats Wichmann
On 12/16/2017 02:12 PM, Bill Deegan wrote:
> Mats,
>
> I see you're filing a bug for this.
> If you check the bug filing policy we ask that you wait for someone to
> approve filing a new bug to prevent duplicates, etc from filling up the bug
> tracker.
>
> Have you tried the lastest changes in the master branch?
> There are some new Java fixes in there since 3.0.1..

I've checked for duplication, but this is effectively new code. pre-3.0
code in jar.py did not do anything like this. The post 3.0.1 jar.py
changes help other problems I have, but not this one.

The problem is in a simple check, os.path.isfile(), which is not safe to
do in the context of a variant dir (without copying), because the source
files in the working/variant directory exist only in the mind of SCons,
not Python. As noted in the bug, I don't at the moment know how to apply
a transformation to convert back to the source directory in the context
in which the tool runs, so figured it was best to document it.

http://scons.tigris.org/issues/show_bug.cgi?id=3042

_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Bill Deegan-2
Did you try the current head of the master branch?


On Sat, Dec 16, 2017 at 5:19 PM, Mats Wichmann <[hidden email]> wrote:
On 12/16/2017 02:12 PM, Bill Deegan wrote:
> Mats,
>
> I see you're filing a bug for this.
> If you check the bug filing policy we ask that you wait for someone to
> approve filing a new bug to prevent duplicates, etc from filling up the bug
> tracker.
>
> Have you tried the lastest changes in the master branch?
> There are some new Java fixes in there since 3.0.1..

I've checked for duplication, but this is effectively new code. pre-3.0
code in jar.py did not do anything like this. The post 3.0.1 jar.py
changes help other problems I have, but not this one.

The problem is in a simple check, os.path.isfile(), which is not safe to
do in the context of a variant dir (without copying), because the source
files in the working/variant directory exist only in the mind of SCons,
not Python. As noted in the bug, I don't at the moment know how to apply
a transformation to convert back to the source directory in the context
in which the tool runs, so figured it was best to document it.

http://scons.tigris.org/issues/show_bug.cgi?id=3042

_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users


_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Mats Wichmann
On 12/16/2017 03:45 PM, Bill Deegan wrote:
> Did you try the current head of the master branch?

yes. I thought that was implied in part of my reply:

>>  The post 3.0.1 jar.py
>> changes help other problems I have, but not this one.
_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Daniel Moody
Hey Mats,

I am looking into this issue, where does the MANIFEST.MF file exist at the start of the build relative to the Scons file you mentioned above? 

What is your VariantDir call look like?

Best regards

On Sat, Dec 16, 2017 at 6:13 PM, Mats Wichmann <[hidden email]> wrote:
On 12/16/2017 03:45 PM, Bill Deegan wrote:
> Did you try the current head of the master branch?

yes. I thought that was implied in part of my reply:

>>  The post 3.0.1 jar.py
>> changes help other problems I have, but not this one.
_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users


_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Daniel Moody
Can you check out the changes from this pull request and see if it fixes your issue?


On Dec 29, 2017 1:56 AM, "Daniel Moody" <[hidden email]> wrote:
Hey Mats,

I am looking into this issue, where does the MANIFEST.MF file exist at the start of the build relative to the Scons file you mentioned above? 

What is your VariantDir call look like?

Best regards

On Sat, Dec 16, 2017 at 6:13 PM, Mats Wichmann <[hidden email]> wrote:
On 12/16/2017 03:45 PM, Bill Deegan wrote:
> Did you try the current head of the master branch?

yes. I thought that was implied in part of my reply:

>>  The post 3.0.1 jar.py
>> changes help other problems I have, but not this one.
_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users


_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Mats Wichmann
On 12/29/2017 12:47 AM, Daniel Moody wrote:
> Can you check out the changes from this pull request and see if it fixes
> your issue?
>
> https://github.com/SCons/scons/pull/35

I will try this presently.  I was exploring something down this path
earlier with the same thinking.  My experiments indicated that the
conversion to File fails only if the string refers to an existing
directory (i.e. you still get a node even if the string does not refer
to an existing file), so while it looks likely to work for our case of
having a manifest file plus already-generated classfiles, it would
prevent the code from hitting the later part of the loop - lines 206 and
later in your patched version.  I don't have proof of that conjecture at
this point, and will update here when I get a chance to test.

>
> On Dec 29, 2017 1:56 AM, "Daniel Moody" <[hidden email]> wrote:
>
>> Hey Mats,
>>
>> I am looking into this issue, where does the MANIFEST.MF file exist at the
>> start of the build relative to the Scons file you mentioned above?
>>
>> What is your VariantDir call look like?
>>
>> Best regards
>>
>> On Sat, Dec 16, 2017 at 6:13 PM, Mats Wichmann <[hidden email]> wrote:
>>
>>> On 12/16/2017 03:45 PM, Bill Deegan wrote:
>>>> Did you try the current head of the master branch?
>>>
>>> yes. I thought that was implied in part of my reply:
>>>
>>>>>  The post 3.0.1 jar.py
>>>>> changes help other problems I have, but not this one.
>>> _______________________________________________
>>> Scons-users mailing list
>>> [hidden email]
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>
>>
>
>
>
> _______________________________________________
> Scons-users mailing list
> [hidden email]
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>

_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users
Reply | Threaded
Open this post in threaded view
|

Re: more Java scons3 issues...

Daniel Moody
Hey mats, 

I think you might be right, actually the last part was to catch the nodes that passed through python isfile or isdir checks previously, so now it may not be needed.

There are cases where it gets passed the two trys added in the PR which I can tell from the swig tests which hit the ".i" extension check, but I am going to investigate the ordering of checks in that PR because it doesn't seem like it should be able to get passed the two tries.

Best regards



On Dec 29, 2017 12:53 PM, "Mats Wichmann" <[hidden email]> wrote:
On 12/29/2017 12:47 AM, Daniel Moody wrote:
> Can you check out the changes from this pull request and see if it fixes
> your issue?
>
> https://github.com/SCons/scons/pull/35

I will try this presently.  I was exploring something down this path
earlier with the same thinking.  My experiments indicated that the
conversion to File fails only if the string refers to an existing
directory (i.e. you still get a node even if the string does not refer
to an existing file), so while it looks likely to work for our case of
having a manifest file plus already-generated classfiles, it would
prevent the code from hitting the later part of the loop - lines 206 and
later in your patched version.  I don't have proof of that conjecture at
this point, and will update here when I get a chance to test.

>
> On Dec 29, 2017 1:56 AM, "Daniel Moody" <[hidden email]> wrote:
>
>> Hey Mats,
>>
>> I am looking into this issue, where does the MANIFEST.MF file exist at the
>> start of the build relative to the Scons file you mentioned above?
>>
>> What is your VariantDir call look like?
>>
>> Best regards
>>
>> On Sat, Dec 16, 2017 at 6:13 PM, Mats Wichmann <[hidden email]> wrote:
>>
>>> On 12/16/2017 03:45 PM, Bill Deegan wrote:
>>>> Did you try the current head of the master branch?
>>>
>>> yes. I thought that was implied in part of my reply:
>>>
>>>>>  The post 3.0.1 jar.py
>>>>> changes help other problems I have, but not this one.
>>> _______________________________________________
>>> Scons-users mailing list
>>> [hidden email]
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>
>>
>
>
>
> _______________________________________________
> Scons-users mailing list
> [hidden email]
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>

_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
[hidden email]
https://pairlist4.pair.net/mailman/listinfo/scons-users