https://pulumi.com logo
Join Slack
Powered by
Has anyone here successfully declared a Pinecone I...
# general
m
Has anyone here successfully declared a Pinecone Index in gcp-starter pod using Pulumi? I'm able to do it with the pinecone client, and through the GUI no problem, but the provider has been an uphill battle.
e.g. this works:
Copy code
import os

from dotenv import load_dotenv
from pinecone import Pinecone, PodSpec  # pylint: disable=import-error

load_dotenv()

pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))

pc.create_index(
    name="my-index", dimension=1536, metric="cosine", spec=PodSpec(environment="us-west-1-gcp", pod_type="p1.x1", pods=1)
)
But many iterations of this, an I'm getting 404s:
Copy code
import pinecone_pulumi as pinecone

# Create a Pinecone Index with a "starter" pod configuration
pinecone_index = pinecone.PineconeIndex("my-index",
    name="my-index",
    spec=pinecone.PineconeSpecArgs(
        pod=pinecone.PineconePodSpecArgs(
            pods=1,
            shards=1,
            pod_type="p1.x1",
            replicas=1,
            environment=pinecone.ServerlessSpecCloud.GCP,
        )
    ),
    metric="cosine",
    dimension=1536,
)
m
@millions-furniture-75402 can you please run 
pulumi up -d
I need to work on “detailed” message output 😅
m
debug: Pinecone index creation response: {"error": {"code":"NOT_FOUND","message":"Resource gcp not found"},"status":404}
m
You are mixing 
environment=pinecone.ServerlessSpecCloud.GCP,
with a pod based index
m
Ahh, good looking out. I changed that to "gcp-starter" and "us-west-1-gcp" and get https://pastebin.com/0LNLLvTe
m
also run with -d for some more infos
Especially the 
Pinecone index creation response: …
is important for me
m
I've been experimenting with the 
pinecone-client
directly The region should be 
us-central1-gcp
but I can't overcome this issue: "Increase you quota or upgrade to create more indexes." If I list indexes (via library or web application), I don't see any. I've tried deleting and recreating the project.
m
do you have a index?
m
no
m
in a different project?
have a look in the UI there is a dropdown
m
Not that I can see. Maybe a bug in their service.
m
otherwise have a look to increase quota
m
I was hoping to do with with the free plan for demonstration purposes.
m
maybe create a new account
m
At this point, it might be easier to host my own vector db 😅
m
temporary
until it’s sorted out
m
Will experiment more later. I can get by without this declaration for the moment.
Oh... interesting... deleting the default project might have been the bug, I cannot set my pod limit above 0 now.
m
interesting
m
Will reach out to their support if I find the bug is on their end.
appreciate your help
m
np, as pinecone is not us i am only here to rubberduck debug with you
m
ah ha, so that wasn't quite the error, but it did give me the hint. 
pod_type
needs to be 
starter
and 
environment
needs to 
gcp-starter
This isn't even documented in Pinecone docs, let alone the Pulumi provider. I was able to infer this by manually creating one in the console. A real missed opportunity for Pinecone.
Further bad news, I can create successfully with the script using the client directly, but those values don't work with the Pulumi declaration.
m
what is the error with -d ?
m
ValueError: '' is not a valid ServerlessSpecCloud
seems to be the issue I can put together an issue for their GitHub repository later this week.
m
but starter is not a serverless
hmm strange
m
Right
m
can you c&p the full code here?
i have a look and play around with, could be an easy fix in the provider!
m
In trying to provide you with a minimum reproducible example, I found the issue, back to "pods". I had 
pod=1
, which ultimately led to the issue. Here's the final "this works with the free tier" example.
Copy code
pinecone_index = pinecone.PineconeIndex(
    "my-index",
    name="my-index",
    spec=pinecone.PineconeSpecArgs(
        pod=pinecone.PineconePodSpecArgs(
            environment="gcp-starter",
            pod_type="starter",
            replicas=1,
        )
    ),
    metric="cosine",
    dimension=1536,
)
Again... I spoke to soon, refresh fails with that. Still some bug.
m
and -d
try to run with -d always
m
It was the ServerlessSpec issue again, sorry currently working on a different computer than I'm talking to you on.
m
kk, let me check! I think that and the snippet should be fine to try
could be a change in the API which is not reflected in the provider
@millions-furniture-75402 for me it runs perfect through. But I used TS, if its matters
Copy code
const myPineconeIndex = new pinecone.PineconeIndex("myPineconeIndex", {
    name: "example-index",
    metric: pinecone.IndexMetric.Cosine,
    spec: {
        pod: {
            podType: "starter",
            replicas: 1,
            pods: 1,
            environment: "gcp-starter",
        }
    },
});
export const output = {
    value: myPineconeIndex.host,
};
m
I've been using Python. I have not tried it with typescript.
m
let me check! one moment
i think I have it
i will release a new version with better error logging
You should now see an message like this: 
error: failed to create Pinecone index: example-index3 with error: Projects only support a single Starter index.
This happens, when you edit an existing resource which has to be replaced to make the change (like remove pod property) You can use https://www.pulumi.com/docs/concepts/options/deletebeforereplace/ to have the index deleted before recreation! new version is 
0.4.1
m
I'll update you when I confirm.
m
Yep, keep an eye on the message now, in case you alter an existing index. I tried to create and delete a classsic, serverless and starter index and all worked smooth. Edit not.
m
It "works" but has a failure the, 
ValueError '' is not a valid ServerlessSpecCloud
. It might have to do with the remarshaling of outputs. I tried recreating my venv, and have the same failure. Looking at the enums in the library now.
m
i need the full stack
plus -d
i am out to grab food and try in pyton
cant get my head around what the issue is
m
give me a moment
sent
m
ok i have it in python too the error!
nice find!
Congratulation @millions-furniture-75402, you found a potential bug in the provider! https://github.com/pulumi/pulumi-go-provider/issues/204 Where can I send you swag? 😄 I should definitely use more python! 😄
new version 
v0.4.2
Open in Slack
PreviousNext
Powered by