https://pulumi.com logo
Join Slack
Powered by
Hi <@U01QK8YMLLT>, <@U05NAUXQ3N0> has some questio...
# java
m
Hi @enough-garden-22763, @thankful-match-46035 has some question regarding the generation of the Java SDK. Can you help us, as she is planning to do some talks around the Java usage of the OVH provider!
cc: @limited-rainbow-51650
s
yes I work with Aurélie and this is the reason for my previous question 😉
m
ah @steep-flower-90579, sorry I did not see the message. My fault. But nevertheless, we are here to help!
t
FYI @enough-garden-22763 we have a working Pulumi provider with a generation of several sdk: go, nodeJS, python and .net. And now we want to generate a Java sdk 🙂 Is it possible to help us at this point? Thanks
l
@thankful-match-46035 does OVH already have an account on the Sonatype Central Portal (or the legacy way) to publish Java packages? https://central.sonatype.org/register/central-portal/
t
No we don't
l
OK, please register first via the Central Portal link. This account can be used to publish any Java packages (not only the Pulumi ones) for OVH.
m
or public OVH artifactory?
l
@many-telephone-49025 that would work too, but then you expect the users to configure an additional repository first before fetching a package. The Central Repository is the default one for Maven/Gradle in the Java ecosystem, similar to npmjs.org for JS/TS and Pypi for Python.
t
We have our internal artifactory, not a public one I think So the first thing should be to have an account in central repository, ok We need to ask internally. What will be then the next step?
l
Next step is to extend the build & publishing workflows for your provider to include Java.
To publish SDKs, we have a reusable Github Actions step which can be used like this:
Copy code
- name: Publish SDKs
      uses: pulumi/pulumi-package-publisher@v0.0.13
      with:
        sdk: all
If you want to see how this composite action works, here is the backing repository for it: https://github.com/pulumi/pulumi-package-publisher Feel free to include me as reviewer when you create the PR in your repository to activate this.
m
@enough-garden-22763 is there something to keep in mind in the 
resources.go
to modify the gradle file to reflect the repo location
e
Thank you for this question! It is definitely possible to publish Java SDKs for providers and it is something my team does for all the providers. Unfortunately streamlining this to GA-level of ease and support has not been prioritized for this quarter so setting this up may take a little more time and patience than would be ideal. I really appreciate community help here to consolidate a README at least for now, in the meanwhile perhaps I can help answering questions and unblocking small bugs. Some notes on how we pubish Java packages internally for Pulumi maintained providers: - we use JAVA_SIGNING_KEY, JAVA_SIGNING_KEY_ID, JAVA_SIGNING_PASSWORD, OSSRH_PASSWORD, OSSRH_USERNAME environment variables to configure Java singning and pubilshing, from https://central.sonatype.org/publish/publish-guide/ - generate Java SDK to commit alongside other languages https://github.com/pulumi/pulumi-random/blob/master/Makefile#L45 and build it with Gradle - the build step publishes an artifact for GitHub actions java-sdk.tar.gz: https://github.com/pulumi/pulumi-random/actions/runs/7586773288/job/20665772861#step:21:48 - publish Java SDK to Maven Central https://github.com/pulumi/pulumi-random/blob/master/.github/workflows/release.yml#L278 which expects an artifact: Artifact java-sdk.tar.gz was downloaded to /home/runner/work/pulumi-random/pulumi-random/sdk There is a known issue that the publishing step is flaky we have not resolved yet. We retry the GitHub action until it succeeds typically.
https://github.com/pulumi/pulumi-random/blob/master/sdk/java/build.gradle#L1 this is an example of the autogenerated gradle file (you can also opt out of autogeneration and write/edit your own). There's a few assumptions here you might need to change like 
groupId = "com.pulumi"
- @many-telephone-49025 which 'repo location' specifically you would like to override?
Also curious if we can find any examples of community maintained providers that would help demonstrate how this works specifically for the community case (CC @big-architect-71258)
b
@enough-garden-22763 My Cookiecutter Template has full support for generating a JAVA SDK for a provider including GitHub Workflows. Currently all steps are commented or not connected to 
build_sdks
because Pulumiverse does not support publishing Java Packages in Maven. Makefile: https://github.com/tmeckel/pulumi-tf-provider-cookiecutter/blob/master/%7B%7Bcookiecutter.provider%7D%7D/Makefile
Copy code
JAVA_GEN         := pulumi-java-gen
JAVA_GEN_VERSION := {{ cookiecutter.__pulumi_java_version }}

....

build_java:: PACKAGE_VERSION := $(shell pulumictl get version --language generic)
build_java:: $(WORKING_DIR)/bin/$(JAVA_GEN)
	$(WORKING_DIR)/bin/$(JAVA_GEN) generate --schema provider/cmd/$(PROVIDER)/schema.json --out sdk/java  --build gradle-nexus
	cd sdk/java/ && \
		echo "module fake_java_module // Exclude this directory from Go tools\n\ngo 1.17" > go.mod && \
		gradle --console=plain build

$(WORKING_DIR)/bin/$(JAVA_GEN)::
	$(shell pulumictl download-binary -n pulumi-language-java -v $(JAVA_GEN_VERSION) -r pulumi/pulumi-java)
Release Workflow https://github.com/tmeckel/pulumi-tf-provider-cookiecutter/blob/master/%7B%7Bcookiecutter.provider%7D%7D/.github/workflows/release.yml
Copy code
- if: ${{ matrix.language == 'java'}}
        name: Setup Java
        uses: actions/setup-java@v3
        with:
          cache: gradle
          distribution: temurin
          java-version: ${{matrix.javaversion}}
Pull Request validation workflow https://github.com/tmeckel/pulumi-tf-provider-cookiecutter/blob/master/%7B%7Bcookiecutter.provider%7D%7D/.github/workflows/pull-request.yml
Copy code
- if: ${{ matrix.language == 'java'}}
      name: Setup Java
      uses: actions/setup-java@v3
      with:
        cache: gradle
        distribution: temurin
        java-version: ${{matrix.javaversion}}

    - name: Build SDK
      run: make build_${{ matrix.language }}
e
Pulumiverse does not support publishing Java Packages in Maven.
Curious do we have this gap documented/tracked somewhere, something I can help with perhaps?
b
@enough-garden-22763 Nope. Last time I spoke about this with @limited-rainbow-51650 there's no account for Pulumiverse in Maven only for Pulumi managed providers.
e
Ah I see what you mean, so there is no shared account and every provider author needs to get their own. Thank for explaining that!
l
@enough-garden-22763 there is no central Pulumiverse account yet. I need to complete the setup.
b
Yes that's right. All other package registries are connected via centrally managed accounts by the Pulumiverse Organization.
Same time 😄
m
@steep-flower-90579 let me know if @enough-garden-22763 links helps you to move forward! Otherwise, we could look to get a call organised (did this with @thankful-match-46035 too) to help unblock you! Would be awesome to get delivered!!
s
@many-telephone-49025 ouahou lot of information! Thanks a lot. I will take time to read all of this and come back to you to tell if need another information. 😊
Hi, First of all, thank for all your messages. I took the time to read them all and try to start my journey to generate a Java SDK for the OVHcloud provider. As I understand it, there are two main steps: - generate the source code files - push into the maven central repository I’m in contact with internal team to create an enterprise account on Sonatype to push the SDK but in parallel I’ll try to generate locally the SDK with the make file before launching the GH action. During this step I get an error :
Copy code
make build_java
error: open /Users/sphilipp/dev/ovh/pulumi-ovh/bin/pulumi-language-java-v0.8.0-darwin-arm64.tar.gz: no such file or directory
/Users/sphilipp/dev/ovh/pulumi-ovh/bin/pulumi-java-gen generate --schema provider/cmd/pulumi-resource-ovh/schema.json --out sdk/java  --build gradle-nexus
make: /Users/sphilipp/dev/ovh/pulumi-ovh/bin/pulumi-java-gen: No such file or directory
make: *** [build_java] Error 1
I don’t know how to install the pulumi-java-gen generate CLI. The make file is on our public repository: https://github.com/ovh/pulumi-ovh/blob/5a2760705b1223a76e859c0596cf9e8a4e10d86f/Makefile#L99-L107 Do you have any idea what’s wrong?
e
https://github.com/pulumi/pulumi-java/releases/tag/v0.9.9 it's released via GitHub so that'd be a good place to fetch it from. In our makefiles we apparently use the pulumictl utility to do so
Copy code
bin/pulumi-java-gen: .pulumi-java-gen.version
	pulumictl download-binary -n pulumi-language-java -v v$(shell cat .pulumi-java-gen.version) -r pulumi/pulumi-java
s
My bad I launched the 
make build_java
instead `make build`` which installs the tooling 😅. First step ✅, source code is generated. I’ll check the next steps to push the JAR :)
Now I’ll check our GH action to compare it to your examples @enough-garden-22763 to create a JAR and publish it on Maven Central
🎉 First successful local test 😊, many thanks for your help. Next step: update the GH action to add java support in SDK generation and release it to Maven Central (and in the Pulumi’s registry 😉)
Copy code
Updating (dev)

View in Browser (Ctrl+O): <https://app.pulumi.com/wildagsx/pulumi-java/dev/updates/4>

     Type                 Name             Status     
     pulumi:pulumi:Stack  pulumi-java-dev             

Outputs:
  + version: "1.26"

Resources:
    1 unchanged

Duration: 4s
m
wow @steep-flower-90579, awesome!
t
💪💪💪
s
Hi (again 😁), I have a question about the gradle file generation. When I run a 
make build_java
to generate the Java SDK file, the code source and the gradle configuration are generated. However my 
gradle.build
file have its fields 
artifactId
and 
groupId
empty:
Copy code
publishing {
    publications {
        mainPublication(MavenPublication) {
            groupId = ""
            artifactId = ""
            version = resolvedVersion
            from components.java
            artifact sourcesJar
            artifact javadocJar

            pom {
                inceptionYear = ""
                name = ""
                packaging = "jar"
                description = ""
How can I configure these fields?
b
@steep-flower-90579 according to Pulumi-Java (https://github.com/pulumi/pulumi-java/blob/main/pkg/codegen/java/build.gradle.template) they are filled from the properties 
.ArtifactID
and 
.GroupID
(screenshot). While generating the 
gradle.build
from the template, the values are copied over from the package info (https://github.com/pulumi/pulumi-java/blob/f7468e19e755561d2a8770ebcc0374245c424b29/pkg/codegen/java/templates_gradle.go#L87). There has been an (already closed) issue about this: https://github.com/pulumi/pulumi-java/issues/1118. So ensure that you are on the latest 
pulumi-java
version v0.9.9.
s
good point @big-architect-71258 my pulumi-java version was 0.8.0, after upgrade it to 0.9.9 my fields have the right values 😉 Thanks!
Hi, I think I’m facing two bugs. Let me know if you prefer me to open issues in the pulumi-java repo or if we continue to exchange here. 1- In some classes, I have a comment that explains that I can use an environment variable to set my public cloud id project but this variable is never read. For example: `The id of the public cloud project. If omitted, the 
OVH_CLOUD_PROJECT_SERVICE
environment variable is used.`. I think this comment is directly imported from our TF provider but the code is not right. 2- the generated version.txt file package is hard coded with 
com/pulumi
but the gradle build generates it in 
com/ovh/ovh
according to the configuration, I think it’s related to the opened issue https://github.com/pulumi/pulumi-java/issues/1110 Thanks!
t
The second point is blocking for us Is it possible to have a little help? thanks 🙂 @limited-rainbow-51650
l
@echoing-dinner-19531 do you know what generates the 
version.txt
file? Is that 
pulumi-java
?
e
I think it's part of the build.gradel file
s
Do you want me to open an issue to explain the bug? At this time I stopped my plan to have the java version of the OVHcloud provider. The "auto update" is very useful to have the Java version and I can't imagine maintaining the Java version manually 😊. If you need more information feel free to ask.
e
Hi Stephane, sorry for dropping the ball here, definitely would be wonderful to file a bug. Even if our maintainers are too busy to look at it right away this helps coordinate with the community.
s
I think you already know this bug since 2023, May 😅: https://github.com/pulumi/pulumi-java/issues/1110 Do you plan to fix this issue or not?
hi, @enough-garden-22763 / @limited-rainbow-51650 do you have news about my issues? It’s the last step for me before doing some talks, videos, ... with Pulumi and OVHcloud
m
Let me check with @enough-garden-22763 an @limited-rainbow-51650 too
e
Hi @steep-flower-90579 I had a quick look but don't have cycles to fix this. As a workaround you might be able to opt out of the Gradle build file generation and edit the Gradle build files to do what is necessary. Tagging @freezing-appointment-56309 for further assistance.
s
ok thanks, juste to be sure Java is still a target for you?
f
Hi @steep-flower-90579 We do not currently have a GA date set, in part do to light customer feedback on the preview. But we do expect to bring this work to GA within the next year. If there are any particular areas of the Java support that are specifically under-developed for your needs, please do let us know!
s
Ok thanks, I’ll continue to use the Java version as I’m a Java developer 😉. I’ll try to do the modifications as @enough-garden-22763 mention for the gradle file and wait that you do some updates on the Java features in futures releases 😉. I may ask 2-3 questions here if I see some issues 😊
f
Yes please do ! 🙂 thank you !
b
@steep-flower-90579 @enough-garden-22763 @thankful-match-46035 @freezing-appointment-56309 working on a fix for the mentioned issue. Still in draft, because I need to provide at least one regression test. But I wanted to inform you guys that some (potential) fix is on the way. https://github.com/pulumi/pulumi-java/pull/1362
f
This is great. thank you for the heads-up !
s
Thanks!!!
b
@steep-flower-90579 @thankful-match-46035 @freezing-appointment-56309 filed a PR https://github.com/pulumi/pulumi-java/pull/1365 to get the long standing issue https://github.com/pulumi/pulumi-java/issues/1109 solved. CC: @enough-garden-22763 @damp-airline-38442
s
Thanks! I'll test it when it will be released 😉
I tested it this morning and everything worked fine, thanks for the job I can move on to release the Java version of the OVHcloud provider 💪
2 Views
Open in Slack
PreviousNext
Powered by