# Authentication Source: https://docs.powerdrill.ai/api-reference/authentication Get your API keys before working on Powerdrill Enterprise *** The Powerdrill Enterprise Open API uses API keys for authentication. Each API request must include your API key, which is used to authenticate the request and track your usage quota. There are two types of API keys: * **Project API keys**: provide access to a specific project. A project's API key can only be used to access resources within that project. * **Team API keys**: provide access to manage projects, such as creating new projects. Only the team admin account can manage projects. Since project-related API endpoints are still under development, team API keys are currently non-functional. As the team admin, you can [manage your projects](/enterprise/projects#manage-your-projects) through the admin console instead. Please be reminded that **your API key is a secret**. Do not share it with others or expose it in browsers, apps, or other client-side code. All API requests must include your API key in the `x-pd-api-key` HTTP header, as shown below: ```shell theme={null} x-pd-api-key: API_KEY ``` *** ## Making requests You can copy and paste the command below into your terminal to execute your first API request. Be sure to replace `$PROJECT_API_KEY` with your secret API key and `$USER_ID` with your actual user ID. ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v1/team/sessions \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PROJECT_API_KEY' \ --data '{ "title": "My session", "languageType": "AUTO", "jobMode": "AUTO", "maxMessagesInContext": 10, "userId": "$USER_ID" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v1/team/sessions" payload = { "title": "My session", "languageType": "AUTO", "jobMode": "AUTO", "maxMessagesInContext": 10, "userId": "$USER_ID" } headers = { "x-pd-api-key": "$PROJECT_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` # Create data source Source: https://docs.powerdrill.ai/api-reference/create-data-source post /v1/team/datasets/{datasetId}/datasources Creates a data source in the specified dataset. # Create data source without specifying a dataset Source: https://docs.powerdrill.ai/api-reference/create-data-source-without-dataset post /v1/team/datasources Creates a data source without specifying a dataset. Powerdrill will automatically create a dataset for it. # Create dataset Source: https://docs.powerdrill.ai/api-reference/create-dataset post /v1/team/datasets Creates a dataset. A dataset is a collection of data sources organized for specific purposes. You can create multiple datasets to group and store data sources based on different needs. # Create job Source: https://docs.powerdrill.ai/api-reference/create-job post /v1/team/jobs Converses with your data. Ask any question you have about your data and get insights instantly. On Powerdrill, a **job** refers to a task that Powerdrill performs to generate a response based on your request (e.g., a prompt or other workflows). Powerdrill Enterprise supports two types of jobs: **general jobs** and **data agent jobs**. Currently, only general jobs are available, so make sure the **x-pd-api-agent-id** header is set to **GENERAL**. - For an in-depth explanation of **jobs**, see [What Is Job?](/enterprise/what-is-job). - To quickly create and execute a job, see [Quick Start](/api-reference/quick-start-for-general). The **Response** section in this topic describes the structure of the response when `stream` is set to `false`. For an example of the response when `stream` is set to `true` and an explanation of how to interpret the streaming response, refer to [Streaming](/api-reference/streaming). # Create session Source: https://docs.powerdrill.ai/api-reference/create-session post /v1/team/sessions Creates a session. A session refers to the continuous interaction between the user and Powerdrill within a conversation. Your team can have up to 150 sessions simultaneously. If the limit is reached, you can remove unnecessary sessions to make space for new ones. # Delete data source Source: https://docs.powerdrill.ai/api-reference/delete-data-source delete /v1/team/datasets/{datasetId}/datasources/{datasourceId} Deletes the specified data source from the specified dataset. Once deleted, the data source cannot be recovered. You can only delete data sources that you have created. # Delete dataset Source: https://docs.powerdrill.ai/api-reference/delete-dataset delete /v1/team/datasets/{datasetId} Deletes the specified dataset. Once deleted, all data sources within the dataset will also be permanently removed and cannot be recovered. # Delete session Source: https://docs.powerdrill.ai/api-reference/delete-session delete /v1/team/sessions/{sessionId} Deletes a session. Once deleted, both the session and its job history will be permanently removed and cannot be recovered. # Error Codes Source: https://docs.powerdrill.ai/api-reference/error-codes Description and troubleshooting suggestions for the error codes you may encounter on our platform Powerdrill Enterprise uses standard HTTP response codes to indicate the result of an API request. In general, codes in the `2xx` range indicate success; codes in the `4xx` range indicate client-side errors (for example, invalid arguments); codes in the `5xx` range indicate server-side errors (these are rare). *** ## API errors | Code | Meaning | | :--------------------------------- | :------------------------------------------------------------------------------------- | | `400 Bad Request` | Invalid parameters provided. | | `401 Unauthorized` | Invalid API key. | | `402 Request Failed` | The parameters were valid, but the request couldn't be completed. | | `403 Forbidden` | The API key lacks the necessary permissions to perform this action. | | `404 Not Found` | The requested resource doesn't exist. | | `409 Conflict` | The request conflicts with another (for example, using the same idempotent key). | | `429 Too Many Requests` | Too many requests sent in a short time. We recommend implementing exponential backoff. | | `500, 502, 503, 504 Server Errors` | Something went wrong on Powerdrill's end (rare). | *** ## Error types | Error type | Description | | :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `authentication_error` | An authentication error indicates your API key is invalid, expired, or revoked. This could be caused by a typo, formatting mistake, or a potential security issue. | | `invalid_request_error` | Invalid request errors arise when your request has invalid parameters. | | `internal_server_error` | An internal server error indicates something went wrong on our end while processing your request. This could be due to a temporary issue, a bug, or a system outage. | | `idempotency_error` | Idempotency errors occur when the same Idempotency-Key is used for a request with a different API endpoint or set of parameters than the original request. | | `rate_limit_error` | A rate limit error indicates that your team has reached the assigned limit of 20 API requests per second. | | `permission_error` | You are not authorized to do this operation. (Action: %s, Resource: %s.) | *** ## Client-side error codes | Error code | Error message | HTTP status code | Description | | | :--------- | :----------------------------------------------------------- | :--------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | | `300001` | Invalid parameters *``* | 400 | A required parameter is missing or incorrectly configured. Please verify that all parameters are properly set. | | | `300002` | No permissions *``* | 200 | You do not have the necessary permissions to perform this operation. | | | `300003` | *``* not found | 200 | The specified resource could not be found in your project. Please verify that the resource ID is correct. | | | `300004` | Invalid file extension | 200 | The file extension is not supported by Powerdrill. Supported extensions include **.csv**, **.tsv**, **.md**, **.mdx**, **.json**, **.txt**, **.pdf**, **.pptx**, **.ppt**, **.doc**, **.docx**, **.xls**, and **.xlsx**. | | | `300005` | Empty file | 200 | The file is empty. Please check that you have uploaded the correct file. | | | `300006` | Insufficient storage space | 200 | There is not enough available storage space to upload the file. Please upgrade your [workspace capacity plan](/developer-guides/create-subscription). | | | `300007` | Failed to upload the file | 200 | An error occured while uploading the file. Please check that the file is in a supported format and try again. | | | `300008` | An error occurred while generating the presigned URL | 200 | An error occurred while generating the presigned URL. Check that all parameters are correctly set and try again. | | | `300009` | The number of sessions has reached the upper limit | 200 | The maximum number of sessions has been reached. Please delete any unnecessary sessions and try again. | | | `300011` | Fail to create datasource | 200 | Failed to create the data source due to an internal error. | | | `210020` | Something went wrong during job execution. Please try again. | 400 | Failed to run the job. You can try again later. | | | `210021` | Job quota exceeded | 400 | Insuffient job quota. Upgrade your job plan to increase your job quota. | Insufficient job quota. Upgrade your plan to run more jobs. | | `210022` | Question too long | 400 | The question exceeds the maximum length of 6000 characters. | | | `210023` | Selected files are not all ready | 400 | At least one of the selected files is not synchronized. | | | `210024` | Text too long for TTS service, limit is 5k characters. | 400 | The text entered to convert to audio exceeds the upper limit of 5000 characters. Please make it shorter. | | | `210025` | Too many selected files in the query | 400 | Too many files selected. | | | `210026` | Invalid analysis | 400 | Failed to analyze your data. You can try again later or set `stream` to true to rerun the job. | | *** ## Server-side error code | Error code | Message | HTTP status code | Description | | :--------- | :------------------------------ | :--------------- | :------------------------------------------------------------------------------------------------------------------------------- | | `9999` | Internal server error | `500` | The request could not be processed due to an unknown error. | | `201` | Rate limit reached for requests | `429` | A rate limit error occurs when you exceed your assigned limit. Currently, each team is allowed up to 20 API requests per second. | | `1002` | Expired credentials | `403` | The provided credentials have expired and are no longer valid. You may need to renew or refresh them. | | `1003` | Insufficient authentication | `403` | The provided authentication is insufficient or incomplete. Please ensure all required authentication details are included. | | `1004` | Bad credentials | `403` | The provided credentials are incorrect or malformed. Please verify that all credentials are correct. | # Get data source Source: https://docs.powerdrill.ai/api-reference/get-data-source get /v1/team/datasets/{datasetId}/datasources/{datasourceId} Obtains information about the specified data source. # Get dataset overview Source: https://docs.powerdrill.ai/api-reference/get-dataset-overview get /v1/team/datasets/{datasetId}/overview Obtains the basic information about the dataset, including the keywords, description, and pre-generated questions. # Get dataset status Source: https://docs.powerdrill.ai/api-reference/get-dataset-status get /v1/team/datasets/{datasetId}/status Datasets are stateless. This endpoint allows you to check the current statuses of data sources in the specified dataset to determine if the dataset is ready for running your jobs. A dataset is fully synchronized only when both `invalidCount` and `synchingCount` in the response are `0`. Otherwise, the data sources that are not synchronized cannot be accessed. # Get job history in session Source: https://docs.powerdrill.ai/api-reference/get-job-history get /v1/team/sessions/{sessionId}/history Obtains the job history contained in the specified session. # Get session Source: https://docs.powerdrill.ai/api-reference/get-session get /v1/team/sessions/{sessionId} Obtains information about the specified session. # List data sources Source: https://docs.powerdrill.ai/api-reference/list-data-sources get /v1/team/datasets/{datasetId}/datasources Lists data sources contained in the specified dataset. # List datasets Source: https://docs.powerdrill.ai/api-reference/list-datasets get /v1/team/datasets Lists datasets in your team or of the specified name. # List sessions Source: https://docs.powerdrill.ai/api-reference/list-sessions get /v1/team/sessions Lists sessions in your team. # Modify dataset Source: https://docs.powerdrill.ai/api-reference/modify-dataset post /v1/team/datasets/{datasetId} Modifies information about the specified dataset. # Modify session Source: https://docs.powerdrill.ai/api-reference/modify-session post /v1/team/sessions/{sessionId} Modifies the configuration of the specified session. # Overview Source: https://docs.powerdrill.ai/api-reference/overview Overview of Powerdrill Enterprise API V1 endpoints and capabilities Powerdrill Enterprise provides a robust set of API endpoints for seamless interaction. All endpoints require authentication with your API key. * To learn how to obtain your API key, refer to [Authentication](/api-reference/authentication). * For detailed instructions on using each endpoint, check the respective topic in this reference. This API Reference is for **Powerdrill Enterprise API V1**. If you're using API V2, please switch to the [API Reference for V2](/api-reference/v2/overview). Datasets on Powerdrill are your knowledge bases that bridging AI to your data. Manage your data sources with offline indexing, vector storage and retrieval. Upload your file without the need to create a data source. Create and manage your sessions to converse with your data. Run a job to start analyzing your data. # Presign data source Source: https://docs.powerdrill.ai/api-reference/presign-data-source post /v1/team/datasets/{datasetId}/datasources/{datasourceId}/presign Generates a presigned URL to access the specified data source. # Streaming Source: https://docs.powerdrill.ai/api-reference/streaming Know all about our streaming capability Powerdrill Enterprise Open API supports streaming responses to clients, enabling partial results for specific requests. This functionality is implemented using the [Server-Sent Events (SSE)](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) standard. *** ## How to understand streaming responses The response to each request consists of a series of response blocks. When streaming mode is enabled for a request, Powerdrill will send real-time updates to the client, delivering continuous response blocks as they becomes available. The structure of a response block is as follows: ```json theme={null} { "id": "", "model": "", "choices": [ { "delta": { "content": "" }, "index": 0 } ], "created": 1731664172, "groupId": "", "groupName": "", "stage": "" } ``` Each streaming response block contains the following fields: * `id` and `groupId`: The ID of the group to which the response block belongs. A group in the streaming response is a collection of response blocks. For example, in a general job, each step in the `Analyze` stage is a group, and the entire `Respond` stage is a group. * `content`: The content of the response block, which varies with the block type. For more details, see [Content description](#content-description). * `created`: The timestamp indicating when the content was created. * `groupId`: The ID of the group to which the response block belongs. * `groupName`: The name of the group, such as `Conclusions`. * `stage`: The stage to which the response block belongs. Two stages are available: `Analyze` and `Respond`. ## Content description The value of `content` in each response block varies with the block content type: * When the block content type is `MESSAGE`: The content is a piece of text. * When the block content type is `CODE`: The content is a code snippet in Markdown format. * When the block content type is `TABLE`: The content represents a table, consisting of: * `name`: The `.csv` file name. * `url`: The S3 key or URL to the file. * `expires_at`: The expiration time for `url`. To save the table for future use, make sure to download it before it expires. * When the block content type is `IMAGE`: The content represents an image, consisting of: * `name`: The image name. * `url`: The S3 key or URL to the image. * `expires_at`: The expiration time for `url`. To save the image for future use, make sure to download it before it expires. * When the block content type is `SOURCES`: The content represents the source of the response block, including: * `source`: The file name of the data source. * `datasourceId`: The ID of the data source. * `datasetId`: The ID of the dataset. * `fileType`: The name extension of the data source file. * When the block content type is `QUESTIONS`: The content represents follow-up questions suggested by Powerdrill. Let's see an example. For details about the `POST /v1/jobs` endpoint, see [Create job](create-job). ```python Python request theme={null} import requests import json url = "https://ai.data.cloud/api/v1/job" payload = json.dumps({ "datasetId": "cm3my37en3q36017q7x3hyyf4", "datasourceIdList": [ "cm3myfsfc03jn011csb8wah6p" ], "languageType": "EN", "question": "How do 'Digital Services for Taxpayers' scores vary across economies, and which economy has the most advanced digital services?", "sessionId": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2", "stream": True }) headers = { 'x-pd-api-key': '$PD_API_KEY', 'x-pd-api-agent-id': 'GENERAL', 'Content-Type': 'application/json' } response = requests.request("POST", url, headers=headers, data=payload) print(response.text) ``` The response is simlilar to this: ``` id:2b5cba4a-d8a5-4beb-aef9-a8612828c405 event:TASK data:{"id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","status":"running","stage":"Analyze","properties":{}}},"index":0}],"created":1731664172,"groupId":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","groupName":"Analyze","stage":"Analyze"} id:2b5cba4a-d8a5-4beb-aef9-a8612828c405 event:TASK data:{"id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","status":"running","stage":"Analyze","properties":{"files":""}}},"index":0}],"created":1731664172,"groupId":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","groupName":"Analyze","stage":"Analyze"} id:2b5cba4a-d8a5-4beb-aef9-a8612828c405 event:TASK data:{"id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","status":"done","stage":"Analyze","properties":{"files":""}}},"index":0}],"created":1731664175,"groupId":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","groupName":"Analyze","stage":"Analyze"} id:d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8 event:TASK data:{"id":"d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8","model":"","choices":[{"delta":{"content":{"name":"Data Understanding","id":"d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8","status":"running","stage":"Analyze","properties":{}}},"index":0}],"created":1731664176,"groupId":"d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8","groupName":"Data Understanding","stage":"Analyze"} id:0ca99d5d-20cf-416c-8bda-0b2549dc8733 event:TASK data:{"id":"0ca99d5d-20cf-416c-8bda-0b2549dc8733","model":"","choices":[{"delta":{"content":{"name":"Aggregate the data by year and calculate the total number of deaths for each disaster type across all entities. This will prepare the data for visualization.","id":"0ca99d5d-20cf-416c-8bda-0b2549dc8733","status":"running","stage":"Analyze","properties":{"files":"makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv"}}},"index":0}],"created":1731664176,"groupId":"0ca99d5d-20cf-416c-8bda-0b2549dc8733","groupName":"Aggregate the data by year and calculate the total number of deaths for each disaster type across all entities. This will prepare the data for visualization.","stage":"Analyze"} id:0ca99d5d-20cf-416c-8bda-0b2549dc8733 event:TASK data:{"id":"0ca99d5d-20cf-416c-8bda-0b2549dc8733","model":"","choices":[{"delta":{"content":{"name":"Aggregate the data by year and calculate the total number of deaths for each disaster type across all entities. This will prepare the data for visualization.","id":"0ca99d5d-20cf-416c-8bda-0b2549dc8733","status":"running","stage":"Analyze","properties":{"files":"makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv"}}},"index":0}],"created":1731664176,"groupId":"0ca99d5d-20cf-416c-8bda-0b2549dc8733","groupName":"Aggregate the data by year and calculate the total number of deaths for each disaster type across all entities. This will prepare the data for visualization.","stage":"Analyze"} id:d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8 event:TASK data:{"id":"d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8","model":"","choices":[{"delta":{"content":{"name":"Data Understanding","id":"d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8","status":"done","stage":"Analyze","properties":{}}},"index":0}],"created":1731664176,"groupId":"d9e9d1d4-fb0b-40e3-a2e6-1f606631a4b8","groupName":"Data Understanding","stage":"Analyze"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:IMAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":{"url":"https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/5e3a766c-8d16-4fc0-b06f-7a337196186d.png","name":"Trends of Deaths from Different Disaster Types Over the Years","expiredAt":"2024-11-21T09:50:00.026476Z"}},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"\n\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Chart Description**: The chart illustrates the trend of death tolls over time for various types of disasters.\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Key Trends**:\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":" - Droughts and floods had high death tolls in the early 20th century.\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":" - Droughts and floods had higher death tolls in the early 20th century. \n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":" - Death tolls from other disaster types remained relatively low and stable. \n\n"},"index":0}],"created":1731664201,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"#### Conclusions and Insights\n"},"index":0}],"created":1731664201,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Significant Impact of Droughts and Floods**: In the early 20th century, droughts and floods caused significantly higher death tolls compared to other disasters.\n"},"index":0}],"created":1731664201,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Fluctuations in Earthquakes and Extreme Weather**: These disasters had significant impacts on death tolls in certain years, highlighting the need for enhanced preventive measures."},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:QUESTIONS data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":["Please analyze the distribution of deaths caused by different types of natural disasters across various countries and regions, and generate a corresponding ECharts map chart.","Please compare the changes in death tolls caused by natural disasters across different decades, and generate a corresponding ECharts line chart.","Please analyze the global death toll trend of a specific natural disaster (e.g., earthquakes) and generate a corresponding ECharts bar chart."]},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:TRIGGER data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":{"name":"conclusion_slice","arguments":{"answer":"$answer"}}},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:TASK data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":{"name":"Conclusions","id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","status":"done","stage":"Respond","properties":{}}},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:-1 event:SOURCES data:{"id":"-1","model":"","choices":[{"delta":{"content":[{"id":"1","source":"makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv","datasourceId":"clxin6l9200oo01l1457bolx3","datasetId":"clxin6l8400ok01l1ff2m0s25","fileType":"csv","externalId":"clxin6l0h001901hzxhjaae6q"}]},"index":0}],"created":1731664202,"groupId":"-1","groupName":"","stage":"Analyze"} ``` However, if streaming is disabled, Powerdrill returns the response only after the entire response is ready. Let's use the same request as an example for a clear comparison. The only difference is that `stream` is set to `False`. ```python theme={null} import requests import json url = "https://ai.data.cloud/api/v1/job" payload = json.dumps({ "datasetId": "cm3my37en3q36017q7x3hyyf4", "datasourceIdList": [ "cm3myfsfc03jn011csb8wah6p" ], "languageType": "EN", "question": "How do 'Digital Services for Taxpayers' scores vary across economies, and which economy has the most advanced digital services?", "sessionId": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2", "stream": False }) headers = { 'x-pd-api-key': '$PD_API_KEY', 'x-pd-api-agent-id': 'GENERAL', 'Content-Type': 'application/json' } response = requests.request("POST", url, headers=headers, data=payload) print(response.text) ``` The response looks like this: ````json Example response theme={null} { "code": 0, "data": { "jobId": "job-cm3ikdeuj02zk01l1yeuirt77", "blocks": [ { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(input_0: pd.DataFrame) -> pd.DataFrame:\n '''\n input_0: pd.DataFrame makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv\n '''\n # Group by 'Year' and sum the deaths for each type of disaster\n aggregated_data = input_0.groupby('Year').sum().reset_index()\n \n # Select only the columns related to deaths\n death_columns = [\n 'Deaths - Drought (decadal)', 'Deaths - Flood (decadal)', \n 'Deaths - Earthquake (decadal)', 'Deaths - Extreme weather (decadal)', \n 'Deaths - Extreme temperature (decadal)', 'Deaths - Volcanic activity (decadal)', \n 'Deaths - Wildfire (decadal)', 'Deaths - Glacial lake outburst flood (decadal)', \n 'Deaths - Dry mass movement (decadal)', 'Deaths - Wet mass movement (decadal)', \n 'Deaths - Fog (decadal)'\n ]\n \n # Create a new DataFrame with the aggregated results\n output = aggregated_data[['Year'] + death_columns]\n \n # Rename columns to be more descriptive\n output.columns = ['Decade'] + [col.replace('Deaths - ', '').replace(' (decadal)', '') for col in death_columns]\n \n return output\n\n```", "groupId": "33063572-6e88-4912-8e2d-4166bcc8caee", "groupName": "Analyze the dataset to observe the trend of deaths caused by different types of natural disasters over the past century. This involves aggregating the data by decade and calculating the total number of deaths for each type of disaster to identify any changes in trends.", "stage": "Analyze" }, { "type": "TABLE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/e24b6a5f-fdb8-48ca-ae35-dc91ac8e8ef7.csv", "name": "trend_data.csv", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "33063572-6e88-4912-8e2d-4166bcc8caee", "groupName": "Analyze the dataset to observe the trend of deaths caused by different types of natural disasters over the past century. This involves aggregating the data by decade and calculating the total number of deaths for each type of disaster to identify any changes in trends.", "stage": "Analyze" }, { "type": "IMAGE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/81b75a33-a223-4954-9680-9f397872c8ad.png", "name": "Trend of Deaths from Natural Disasters Over the Century", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "7501680b-5879-441b-bd96-f58b1029ae17", "groupName": "Visualize the trend data to show how the number of deaths from different types of natural disasters has changed over the past century. Use line charts to represent the trends for each disaster type, which will help in understanding the impact of measures and technological advancements on reducing deaths.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "\n\n`Analyzing Conclusions` \n\n### Analysis of Trends in Natural Disaster Fatalities\n\n#### Data Analysis\n\n", "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "TABLE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/e24b6a5f-fdb8-48ca-ae35-dc91ac8e8ef7.csv", "name": "trend_data.csv", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Droughts and Floods**: In the early 20th century, droughts and floods caused extremely high death tolls, particularly in the 1920s and 1930s.\n- **Earthquakes and Extreme Weather**: Earthquakes and extreme weather also led to significant fatalities throughout the century, especially in the 1970s and 1990s.\n- **Extreme Temperatures and Volcanic Activity**: These disasters had relatively lower death tolls, but in certain decades, such as the 2000s, deaths caused by extreme temperatures increased.\n\n#### Trend Visualization\n\n", "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "IMAGE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/81b75a33-a223-4954-9680-9f397872c8ad.png", "name": "Trend of Deaths from Natural Disasters Over the Century", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Overall Trend**: The chart shows that, despite spikes in death tolls caused by natural disasters in certain decades, the overall trend is declining.\n- **Impact of Technology and Measures**: Over time, advancements in technology and the strengthening of disaster prevention measures are likely key factors contributing to the reduction in fatalities.\n\n#### Conclusions and Insights\n- **Technological Advancements**: Progress in modern technology, such as improved early warning systems and construction techniques, may have reduced deaths caused by earthquakes and extreme weather.\n- **Disaster Prevention Measures**: The enhancement of global disaster prevention efforts and emergency response capabilities has likely played a crucial role in mitigating the fatality rates of natural disasters.", "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "SOURCES", "content": [ { "source": "makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv", "datasourceId": "clxin6l9200oo01l1457bolx3", "datasetId": "clxin6l8400ok01l1ff2m0s25", "fileType": "csv", "externalId": "clxin6l0h001901hzxhjaae6q" } ], "groupId": "", "groupName": "", "stage": "Respond" }, { "type": "QUESTIONS", "content": [ "Analyze the changes in death toll trends for different types of natural disasters over the past century and explore which types of disasters have experienced the most significant reductions in fatalities.", "Study the differences in technological advancements and measures for responding to natural disasters across various regions globally, and analyze how these differences have influenced changes in death tolls in each region.", "Explore how potential future technological advancements and policy measures could further reduce fatalities caused by natural disasters, and assess their feasibility and potential impacts." ], "groupId": "-1", "stage": "Respond" } ] } } ```` *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Upload file Source: https://docs.powerdrill.ai/api-reference/upload-file post /v1/team/file/upload_datasource Uploads a file. ### Introduction You can use this endpoint to upload your local file and then use the `fileKey` you obtained to create a data source through the [Create data source](/api-reference/create-data-source) endpoint. Only files with the following extensions are supported: **.csv**, **.tsv**, **.md**, **.mdx**, **.json**, **.txt**, **.pdf**, **.pptx**, **.ppt**, **.doc**, **.docx**, **.xls**, or **.xlsx**. **Example request:** ```curl cURL theme={null} curl --location 'http://ai.data.cloud/api/v1/file/upload_datasource' \ --header 'Content-Type: multipart/form-data' \ --header 'x-pd-api-key: ' \ --header 'x-pd-api-agent-id: GENERAL' \ --form 'file=@"/Users/test/workspace/66C4547D0000650003AB1.xlsx"' ``` ```python Python theme={null} import requests url = "http://ai.data.cloud/api/v1/file/upload_datasource" payload = {} files=[ ('file',('66C4547D0000650003AB1.xlsx',open('/Users/test/workspace/66C4547D0000650003AB1.xlsx','rb'),'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')) ] headers = { 'Content-Type': 'multipart/form-data', 'x-pd-api-key': '', 'x-pd-api-agent-id': 'GENERAL' } response = requests.request("POST", url, headers=headers, data=payload, files=files) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "fileKey": "/tmp/sdgsagdsgsadgasdg" } } ``` # Authentication Source: https://docs.powerdrill.ai/api-reference/v2/authentication Get your API keys before working on Powerdrill Enterprise *** The Powerdrill Enterprise Open API uses API keys for authentication. Each API request must include your API key, which is used to authenticate the request and track your usage quota. There are two types of API keys: * **Project API keys**: provide access to a specific project. A project's API key can only be used to access resources within that project. * **Team API keys**: provide access to manage projects, such as creating new projects. Only the team admin account can manage projects. Since project-related API endpoints are still under development, team API keys are currently non-functional. As the team admin, you can [manage your projects](/enterprise/projects#manage-your-projects) through the admin console instead. Please be reminded that **your API key is a secret**. Do not share it with others or expose it in browsers, apps, or other client-side code. All API requests must include your API key in the `x-pd-api-key` HTTP header, as shown below: ```shell theme={null} x-pd-api-key: API_KEY ``` *** ## Making requests You can copy and paste the command below into your terminal to execute your first API request. Be sure to replace `$PROJECT_API_KEY` with your secret API key and `$USER_ID` with your actual user ID. ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v2/team/sessions \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PROJECT_API_KEY' \ --data '{ "name": "My session", "output_language": "EN", "job_mode": "AUTO", "max_contextual_job_history": 10, "agent_id": "DATA_ANALYSIS_AGENT", "user_id": "$USER_ID" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/sessions" payload = { "name": "My session", "output_language": "EN", "job_mode": "AUTO", "max_contextual_job_history": 10, "agent_id": "DATA_ANALYSIS_AGENT", "user_id": "tmm-dafasdfasdfasdf" } headers = { "x-pd-api-key": "$PROJECT_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` # Complete multipart upload Source: https://docs.powerdrill.ai/api-reference/v2/complete-multipart-upload post /v2/team/file/complete-multipart-upload Completes a multipart upload task after all parts of the corresponding file are uploaded. ## Important notes When calling this endpoint, you must provide a full list of uploaded parts, including their sequential numbers and corresponding ETag values. Cloud Object Storage service (for example, AWS S3) validates each part in sequence and, upon successful verification, combines them into a complete file. # Create data source Source: https://docs.powerdrill.ai/api-reference/v2/create-data-source POST /v2/team/datasets/{id}/datasources Creates a data source in the specified dataset. You can create data sources only in datasets that you have created. A data source can be in one of the following formats: **.csv**, **.tsv**, **.md**, **.mdx**, **.json**, **.txt**, **.pdf**, **.pptx**, **.docx**, **.xls**, or **.xlsx**. # Create data source without specifying a dataset Source: https://docs.powerdrill.ai/api-reference/v2/create-data-source-without-dataset POST /v2/team/datasources Creates a data source without specifying a dataset. When calling this endpoint, Powerdrill will automatically create a dataset for the new data source. Save the dataset ID returned in the response so that you can [associate it with a job](/api-reference/v2/create-job) and explore the data source. # Create dataset Source: https://docs.powerdrill.ai/api-reference/v2/create-dataset post /v2/team/datasets Creates a dataset. A dataset is a collection of related data sources, such as Excel, CSV, PDF, Word, webpages, Markdown, and plain text. It allows easy access to and utilization of your data for analysis. You can create a dataset to host all the data sources needed for a specific analysis. Then, when creating a session, simply associate it with the dataset to access all the required data sources. # Create job Source: https://docs.powerdrill.ai/api-reference/v2/create-job post /v2/team/jobs Converses with your data. Ask any question you have about your data and get insights instantly. On Powerdrill, a **job** refers to a task that Powerdrill performs to generate a response based on your request (e.g., a prompt or other workflows). For an in-depth explanation of **jobs**, see [What Is Job?](/enterprise/what-is-job). # Create session Source: https://docs.powerdrill.ai/api-reference/v2/create-session post /v2/team/sessions Creates a session. A session represents a continuous interaction between a user and Powerdrill within a single conversation. Each user can have up to 150 active sessions simultaneously. If this limit is reached, you can remove unused sessions to free up space for new ones. # Create subscriptions Source: https://docs.powerdrill.ai/api-reference/v2/create-subscription How to create or upgrade your subscriptions to get more quotas This guide is intended for team admins. If you're a system or virtual user, please contact your team admin to have them assign or upgrade the subscription linked to your account. After [creating your team](/enterprise/workspaces#create-a-workspace) on Powerdrill Enterprise, you'll receive 100 free jobs for a one-month trial and 100 MB of permanent free Workspace capacity storage. However, you may need to subscribe to or upgrade your plans when using Powerdrill Enterprise's API in the following cases: * [You have used up your free job quota](#create-subscriptions). Once the 100 free jobs are used or the 1-month trial ends, no job quota remains. Subscribe to a paid plan to continue using Powerdrill Enterprise. * [You want to expand your team](#create-subscriptions). Users in your team can run jobs only when linked to a job subscription, and each subscription can be associated with only one user. To add more users in your team, you need additional job subscriptions. * [Your Workspace capacity is full](#upgrade-your-workspace-capacity-plan). The free Workspace capacity storage is limited to 100 MB. If your team's data reaches this limit, upgrade your Workspace capacity plan to increase storage space. * [The job quota offered by a specific subscription is insufficient](#upgrade-job-subscriptions). If the allocated job quota for a subscription is not enough, you need to upgrade the subscription to meet your usage needs. In this guide, we'll show you how to upgrade your free plan and create additional job subscriptions. *** ## Before you start Please note that: * All subscription-related operations are managed in your Admin console. Read this guide to learn more about [how to enter your Admin console](/enterprise/enter-admin-console.mdx). * For more information about job and Workspace capacity plans, see [Pricing](/enterprise/pricing.mdx). *** ## Create subscriptions 1. On the top navigation bar in the Admin console, select **Subscriptions & plans**. 2. On the page that appears, click **Subscribe**. 3. On the **Select plan** page, choose the subscription type, select the plan, and set the number of subscriptions you want to make, and click **Confirm**. There are two subscription types available: * **Monthly subscription**: Recurs on a monthly basis. * **One-time purchase**: Non-recurring and expires after the set period. When deciding on the number of subscriptions, consider how many users in your team need to run jobs. Ensure that the number of subscriptions matches the number of users in your team. 4. Once the payment is successful, you can manage your subscriptions on the **Subscriptions & plans** page. *** ## Upgrade job subscriptions 1. On the top navigation bar in the Admin console, select **Subscriptions & plans**. 2. In the row of the target subscription, choose **Manage** > **Upgrade** in the **Actions** column. 3. On the page that appears, select the plan you want to upgrade to and click **Upgrade**. *** ## Upgrade your Workspace capacity plan 1. In the Admin console, select **Usage & billing** from the top navigation bar. 2. In the **Workspace capacity** section, click **Upgrade**. 3. On the **Select capacity plan** page, choose your desired plan and click **Upgrade**. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Delete data source Source: https://docs.powerdrill.ai/api-reference/v2/delete-data-source delete /v2/team/datasets/{dataset_id}/datasources/{datasource_id} Deletes a data source from the specified dataset. Once deleted, the data source cannot be recovered. You can only delete data sources from datasets you have created. # Delete dataset Source: https://docs.powerdrill.ai/api-reference/v2/delete-dataset delete /v2/team/datasets/{id} Deletes a dataset that you created. Once deleted, all data sources within the dataset will also be permanently removed and cannot be recovered. # Delete session Source: https://docs.powerdrill.ai/api-reference/v2/delete-session delete /v2/team/sessions/{id} Deletes a session that you created. Once deleted, both the session and its job history will be permanently removed and cannot be recovered. # Get data source Source: https://docs.powerdrill.ai/api-reference/v2/get-data-source get /v2/team/datasets/{dataset_id}/datasources/{datasource_id} Obtains information about the specified data source. **Example request:** ```curl cURL theme={null} curl --request GET \ --url 'https://ai.data.cloud/api/v2/team/datasets/{dataset_id}/datasources/{datasource_id}?user_id=tmm-dafasdfasdfasdf' \ --header 'x-pd-api-key: ' \ --header 'x-pd-external-trace-id: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets/{dataset_id}/datasources/{datasource_id}?user_id=tmm-dafasdfasdfasdf" headers = { "x-pd-api-key": "gegeege", "x-pd-external-trace-id": "" } response = requests.request("GET", url, headers=headers, params=querystring) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "id": "ds-cm7aedl58000501fc52hjj5c3", "name": "ds1.xlsx", "type": "FILE", "status": "synched", "size": 14435, "dataset_id": "dset-cm7ae4d77028n01l1cnapd5si" } } ``` # Get dataset overview Source: https://docs.powerdrill.ai/api-reference/v2/get-dataset-overview get /v2/team/datasets/{id}/overview Obtains the basic information about the dataset, including the keywords, description, and pre-generated questions. **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/datasets/{id}/overview?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets/{id}/overview?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "id": "dset-cm5axptyyxxx298", "name": "sales_indicators_2024", "description": "A dataset comprising 373 travel bookings with 15 attributes, offering insights into booking patterns, pricing strategies, and more", "summary": "This dataset contains 373 travel bookings with 15 attributes, enabling analysis of booking trends, pricing strategies, and travel agency dynamics.", "exploration_questions": [ "How does the booking price trend over time based on the BookingTimestamp?", "How does the average booking price change with respect to the TravelDate?", "Are there any significant outliers in the booking prices, and what might be causing them?", "How does the average price vary between one-way and round-trip bookings?" ], "keywords": [ "Travel Bookings", "Booking Trends", "Travel Agencies" ] } } ``` # Get status summary of data sources in dataset Source: https://docs.powerdrill.ai/api-reference/v2/get-dataset-status get /v2/team/datasets/{id}/status Counts the data sources for each status in a dataset. This endpoint returns the count of data sources for each status in the specified dataset. You can use it to verify if all data sources are synchronized and ready for running data analysis jobs. If both `invalid_count` and `synching_count` in the response are `0`, all data sources in the dataset are accessible to Powerdrill for answering questions. Otherwise, any unsynchronized data sources cannot be accessed. **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/datasets/{id}/status?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets/{id}/status?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "synched_count": 5, "invalid_count": 0, "synching_count": 0 } } ``` # Get job history in session Source: https://docs.powerdrill.ai/api-reference/v2/get-job-history get /v2/team/sessions/{id}/history Obtains the job history retained in the specified session. **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/sessions/{id}/history?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/sessions/{id}/history?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ````json 200 theme={null} { "code": 0, "data": { "total_items": 1, "page_number": 1, "page_size": 10, "records": [ { "job_id": "job-1dsfasddfasgddsaffds", "question": { "blocks": [ { "type": "MESSAGE", "content": "Check for negative values across all sales columns" } ] }, "answer": { "blocks": [ { "type": "MESSAGE", "content": "- Check for negative values across all sales columns.\n- Filter the DataFrame to retain only rows with no negative sales values.", "group_id": "ba582fc9-bb36-4c5d-a8e8-d35bda6389cd", "group_name": "Identify the channels with negative sales values by examining each day's sales data. Filter out the rows where any sales value is negative.", "stage": "Analyze" }, { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(input_0: pd.DataFrame) -> pd.DataFrame:\n '''\n input_0: pd.DataFrame SalesByChannelByDay_Summary_Demo.Sheet1_0_table_1.csv\n '''\n # Identify columns that represent sales data (all except the first column)\n sales_columns = input_0.columns[1:]\n \n # Filter rows where any sales value is negative\n filtered_df = input_0[~(input_0[sales_columns] < 0).any(axis=1)]\n \n # Assign the result to the output variable\n output = filtered_df\n \n return output\n\n```", "group_id": "ba582fc9-bb36-4c5d-a8e8-d35bda6389cd", "group_name": "Identify the channels with negative sales values by examining each day's sales data. Filter out the rows where any sales value is negative.", "stage": "Analyze" }, { "type": "TABLE", "content": "https://s3.amazonaws.com/powerdrilltest/tmp_datasource_cache/code_result/clvl4cad2001q01l1m522hxlu/baf7d6d1-fb81-4fdb-bcdd-32923d214c7b.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20241018T104617Z&X-Amz-SignedHeaders=host&X-Amz-Expires=599&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20241018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=9bcb5af552793f162e35f41d62fb9306cf90888924bfbdce81ea687265fddf83", "group_id": "ba582fc9-bb36-4c5d-a8e8-d35bda6389cd", "group_name": "Identify the channels with negative sales values by examining each day's sales data. Filter out the rows where any sales value is negative.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "- Sum the sales across all days for each channel.\n- Create a new DataFrame with the channel names and their corresponding total sales.", "group_id": "47183fd1-307b-4408-9986-e9238d952ec1", "group_name": "Calculate the overall sales trend for the identified channels with negative sales values. This involves summing up the sales across all days for each channel and analyzing the trend.", "stage": "Analyze" }, { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(negative_sales_channels: pd.DataFrame) -> pd.DataFrame:\n '''\n negative_sales_channels: pd.DataFrame negative_sales_channels.csv\n '''\n # Sum the sales across all days for each channel\n total_sales = negative_sales_channels.iloc[:, 1:].sum(axis=1)\n \n # Create a new DataFrame with the channel names and their corresponding total sales\n output = pd.DataFrame({\n 'Channel': negative_sales_channels.iloc[:, 0],\n 'Total Sales': total_sales\n })\n \n return output\n\n```", "group_id": "47183fd1-307b-4408-9986-e9238d952ec1", "group_name": "Calculate the overall sales trend for the identified channels with negative sales values. This involves summing up the sales across all days for each channel and analyzing the trend.", "stage": "Analyze" }, { "type": "TABLE", "content": "https://s3.amazonaws.com/powerdrilltest/tmp_datasource_cache/code_result/clvl4cad2001q01l1m522hxlu/10cffac2-8bf3-45f4-86e6-1ed8457329f2.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20241018T104617Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20241018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=c6f5b522d2ddceea730304b86a45d5f5165f05f9fda3c1d275d11e9022c1e7ac", "group_id": "47183fd1-307b-4408-9986-e9238d952ec1", "group_name": "Calculate the overall sales trend for the identified channels with negative sales values. This involves summing up the sales across all days for each channel and analyzing the trend.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "- Replace any negative sales values with zero in the data.\n- Sum the sales across all days for each channel.\n- Create a new data structure with the summed sales values.", "group_id": "6b93c2b1-8908-4c2b-afb2-2a81f2d24739", "group_name": "Calculate the overall sales trend for the same channels but excluding the negative sales values. This involves setting negative values to zero or removing them and then summing up the sales across all days for each channel.", "stage": "Analyze" }, { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(negative_sales_channels: pd.DataFrame) -> pd.DataFrame:\n '''\n negative_sales_channels: pd.DataFrame negative_sales_channels.csv\n '''\n # Replace negative values with zero\n negative_sales_channels.iloc[:, 1:] = negative_sales_channels.iloc[:, 1:].clip(lower=0)\n \n # Sum the sales across all days for each channel\n sales_sum = negative_sales_channels.iloc[:, 1:].sum(axis=1)\n \n # Create a new DataFrame with the summed sales values\n output = pd.DataFrame({\n 'Channel': negative_sales_channels.iloc[:, 0],\n 'Total Sales': sales_sum\n })\n \n return output\n\n```", "group_id": "6b93c2b1-8908-4c2b-afb2-2a81f2d24739", "group_name": "Calculate the overall sales trend for the same channels but excluding the negative sales values. This involves setting negative values to zero or removing them and then summing up the sales across all days for each channel.", "stage": "Analyze" }, { "type": "TABLE", "content": "https://s3.amazonaws.com/powerdrilltest/tmp_datasource_cache/code_result/clvl4cad2001q01l1m522hxlu/f4c99616-dd7c-48b1-8a35-d3141d732c36.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20241018T104617Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20241018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=d21a8d939ab09547bc201754ba253ca6c0d1da5361752f2228237e3ff59be256", "group_id": "6b93c2b1-8908-4c2b-afb2-2a81f2d24739", "group_name": "Calculate the overall sales trend for the same channels but excluding the negative sales values. This involves setting negative values to zero or removing them and then summing up the sales across all days for each channel.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "- Merge the two datasets on the 'Channel' column to align sales data for comparison.\n- Calculate the difference in 'Total Sales' between the datasets for each channel.\n- Store the results, including channel name and calculated difference, in a new dataset.", "group_id": "3488f538-f7fc-4c0e-a265-b66e3a38d41e", "group_name": "Compare the sales trends with and without negative sales values to determine the impact of negative sales on the overall sales trend for the affected channels.", "stage": "Analyze" }, { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(sales_trend_with_negatives: pd.DataFrame, sales_trend_without_negatives: pd.DataFrame) -> pd.DataFrame:\n # Merge the two DataFrames on the 'Channel' column\n merged_df = pd.merge(sales_trend_with_negatives, sales_trend_without_negatives, on='Channel', suffixes=('_with_negatives', '_without_negatives'))\n \n # Calculate the difference in 'Total Sales' between the two DataFrames\n merged_df['Sales Difference'] = merged_df['Total Sales_without_negatives'] - merged_df['Total Sales_with_negatives']\n \n # Create a new DataFrame to store the results\n output = merged_df[['Channel', 'Sales Difference']]\n \n return output\n\n```", "group_id": "3488f538-f7fc-4c0e-a265-b66e3a38d41e", "group_name": "Compare the sales trends with and without negative sales values to determine the impact of negative sales on the overall sales trend for the affected channels.", "stage": "Analyze" }, { "type": "TABLE", "content": "https://s3.amazonaws.com/powerdrilltest/tmp_datasource_cache/code_result/clvl4cad2001q01l1m522hxlu/aaf4f2f7-e2db-4f2e-98ae-0bdd18f42333.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20241018T104617Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20241018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=19fcfe97c70bf40292b920ecbf0299c101e1de11e49b54fb8f62934b6e874f52", "group_id": "3488f538-f7fc-4c0e-a265-b66e3a38d41e", "group_name": "Compare the sales trends with and without negative sales values to determine the impact of negative sales on the overall sales trend for the affected channels.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "\n\n`Analyzing Conclusions` \n\n### The impact of negative sales values on overall sales trends\n\n#### Sales variance analysis\n\n", "group_id": "fd1a62e6-48cf-4ac1-8bac-025665444710", "group_name": "Conclusions", "stage": "Respond" }, { "type": "TABLE", "content": "https://s3.amazonaws.com/powerdrilltest/tmp_datasource_cache/code_result/clvl4cad2001q01l1m522hxlu/aaf4f2f7-e2db-4f2e-98ae-0bdd18f42333.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20241018T104617Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20241018%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=19fcfe97c70bf40292b920ecbf0299c101e1de11e49b54fb8f62934b6e874f52", "group_id": "fd1a62e6-48cf-4ac1-8bac-025665444710", "group_name": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Sales variance**:In all channels (including EC, JD, Tmall, WeChat, retail, corporate stores, outlets, and total), the sales difference is 0.0. This indicates that regardless of the presence of negative sales values, the sales trend has not changed.\n\n#### Conclusion and Insights\n- **The impact of negative sales values**:Based on the provided data, negative sales values have no impact on the overall sales trend of the affected channels, as the sales variance for all channels is 0.0.\n- **Data consistency**:The sales discrepancies across all channels are consistent, indicating that there are no anomalies or deviations caused by negative sales values during data processing or analysis.", "group_id": "fd1a62e6-48cf-4ac1-8bac-025665444710", "group_name": "Conclusions", "stage": "Respond" }, { "type": "SOURCES", "content": [ { "source": "SalesByChannelByDay_Summary_Demo.xlsx", "datasource_id": "cm2ej4wmo000001fcdkwbdrml", "dataset_id": "cm2ej4vx900hp01l1o378zr9o", "file_type": "xlsx", "external_id": "" } ], "group_id": "", "group_name": "", "stage": "Respond" }, { "type": "QUESTIONS", "content": [ "Analyze the specific channels with negative sales values on different dates and discuss whether the sales strategies of these channels might lead to negative values.", "Study the long-term impact of negative sales values on overall sales trends and assess whether adjustments to data analysis methods are needed to more accurately reflect the actual situation.", "Investigate the source of negative sales values, whether they are related to returns, discounts, or other factors, and propose possible solutions to reduce the occurrence of negative values." ], "group_id": "-1", "stage": "Respond" } ] } } ] } } ```` # Get session Source: https://docs.powerdrill.ai/api-reference/v2/get-session get /v2/team/sessions/{id} Obtains information about a session you created. **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/sessions/{id}?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/sessions/{id}?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "id": "cckrXXg68P8lmb59dg4yO", "name": "Analyze performance", "output_language": "HI", "job_mode": "AUTO", "max_contextual_job_history": 76, "agent_id": "DATA_ANALYSIS_AGENT" } } ``` # Check Status of Data Sources Source: https://docs.powerdrill.ai/api-reference/v2/how-to-check-data-sources How to verify that all data sources in the specified dataset are synchronized Powerdrill lets you upload multiple data sources to a dataset, making it easier to organize your data files. However, any data sources that are not synchronized at the time a job starts will be excluded from analysis or exploration. To ensure all data sources are used in data analysis, make sure they are fully synchronized before executing data jobs. Here's how: *** ## 1. Call the endpoint Send a request to the [Get status summary of data sources in dataset ](/api-reference/v2/get-dataset-status) endpoint. Following is a request example: Set `id` to the dataset ID, `user_id` to your user ID, and `x-pd-api-key` to the API key of the target project. ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets/{id}/status?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` *** ## 2. Check the result Then, review the returned response: ```json Example response theme={null} { "code": 0, "data": { "synched_count": 5, "invalid_count": 0, "synching_count": 0 } } ``` If both `invalid_count` and `synching_count` are `0`, all data sources in the dataset are fully synchronized and ready to be used by Powerdrill. Otherwise, any unsynchronized sources will not be available for analysis. # Initiate multipart upload Source: https://docs.powerdrill.ai/api-reference/v2/initiate-multipart-upload post /v2/team/file/init-multipart-upload Initiates a multipart upload task for local files in multipart mode. Only files in the following formats are supported: **.csv**, **.tsv**, **.md**, **.mdx**, **.json**, **.txt**, **.pdf**, **.pptx**, **.docx**, **.xls**, or **.xlsx**. ## Important notes When you call this endpoint, Cloud Object Storage, for example, AWS S3, generates and returns a unique upload ID to identify the multipart upload task. This ID is required when you call the [Complete multipart upload](/api-reference/v2/complete-multipart-upload) endpoint. # List data sources Source: https://docs.powerdrill.ai/api-reference/v2/list-data-sources get /v2/team/datasets/{id}/datasources Lists data sources in the specified dataset. When using this endpoint: - Make sure the specified dataset belongs to the same project as your API key. - To check the datasets you have access to in the project, call [GET /v2/team/datasets](/api-reference/v2/list-datasets). **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/datasets/{id}/datasources?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets/{id}/datasources?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "total_items": 1, "page_size": 10, "page_number": 1, "records": [ { "code": 0, "data": { "id": "ds-cm7aedl58000501fc52hjj5c3", "name": "ds1.xlsx", "type": "FILE", "status": "synched", "size": 14435, "dataset_id": "dset-cm7ae4d77028n01l1cnapd5si" } } ] } } ``` # List datasets Source: https://docs.powerdrill.ai/api-reference/v2/list-datasets get /v2/team/datasets Lists datasets. Only datasets created within the same project as your API key will be listed. You can specify a search keyword to filter datasets by name and description. **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/datasets?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "page_number": 1, "page_size": 10, "total_items": 1, "records": [ { "id": "dataset-dasfadsgadsgas", "name": "mysql", "description": "mysql databases" } ] } } ``` # List sessions Source: https://docs.powerdrill.ai/api-reference/v2/list-sessions get /v2/team/sessions Lists sessions that you created. **Example request:** ```curl cURL theme={null} curl --request GET \ --url https://ai.data.cloud/api/v2/team/sessions?user_id=tmm-dafasdfasdfasdf \ --header 'x-pd-api-key: ' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/sessions?user_id=tmm-dafasdfasdfasdf" headers = {"x-pd-api-key": ""} response = requests.request("GET", url, headers=headers) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "page_number": 1, "page_size": 10, "total_items": 1, "records": [ { "id": "dataset-dasfadsgadsgas", "name": "mysql", "description": "mysql databases" } ] } } ``` # Modify dataset Source: https://docs.powerdrill.ai/api-reference/v2/modify-dataset post /v2/team/datasets/{id} Modifies the name or description of specified dataset. # Modify session Source: https://docs.powerdrill.ai/api-reference/v2/modify-session post /v2/team/sessions/{id} Modifies the configuration of a session that you created. # Overview Source: https://docs.powerdrill.ai/api-reference/v2/overview Overview of Powerdrill Enterprise API V2 endpoints and capabilities Powerdrill Enterprise provides a robust set of API endpoints for seamless interaction. All endpoints require authentication with your API key. * To learn how to obtain your API key, refer to [Authentication](/api-reference/v2/authentication). * For detailed instructions on using each endpoint, check the respective topic in this reference. This API Reference is for **Powerdrill Enterprise API V2**. If you're using API V1, please switch to the [API Reference for V1](/api-reference/overview). Datasets on Powerdrill are your knowledge bases that bridging AI to your data. Manage your data sources with offline indexing, vector storage and retrieval. Upload your file without the need to create a data source. Create and manage your sessions to converse with your data. Run a job to start analyzing your data. # Presign data source Source: https://docs.powerdrill.ai/api-reference/v2/presign-data-source post /v2/team/datasets/{dataset_id}/datasources/{datasource_id}/presign If you want to download a data source, you can use this endpoint to generate a presigned URL, which allows you to download the data source via the URL. Presigned URLs have an expiration period. Be sure to download your data sources before the URL expires. # Streaming Source: https://docs.powerdrill.ai/api-reference/v2/streaming Know all about our streaming capability Powerdrill Enterprise Open API supports streaming responses to clients, enabling partial results for specific requests. This functionality is implemented using the [Server-Sent Events (SSE)](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) standard. *** ## How to understand streaming responses The response to each request consists of a series of response blocks. When streaming mode is enabled for a request, Powerdrill will send real-time updates to the client, delivering continuous response blocks as they becomes available. The structure of a response block is as follows: ```json theme={null} { "id": "", "model": "", "choices": [ { "delta": { "content": "" }, "index": 0 } ], "created": 1731664172, "group_id": "", "group_name": "", "stage": "" } ``` Each streaming response block contains the following fields: * `id` and `group_id`: The ID of the group to which the response block belongs. A group in the streaming response is a collection of response blocks. For example, in a general job, each step in the `Analyze` stage is a group, and the entire `Respond` stage is a group. * `content`: The content of the response block, which varies with the block type. For more details, see [Content description](#content-description). * `created`: The timestamp indicating when the content was created. * `group_id`: The ID of the group to which the response block belongs. * `group_name`: The name of the group, such as `Conclusions`. * `stage`: The stage to which the response block belongs. Two stages are available: `Analyze` and `Respond`. When you encounter `:keep-alive` in a response, simply ignore it—it serves only as a heartbeat signal to maintain the connection. *** ## Content description The value of `content` in each response block varies with the block content type: * When the block content type is `MESSAGE`: The content is a piece of text. * When the block content type is `CODE`: The content is a code snippet in Markdown format. * When the block content type is `TABLE`: The content represents a table, consisting of: * `name`: The `.csv` file name. * `url`: The S3 key or URL to the file. * `expires_at`: The expiration time for `url`. To save the table for future use, make sure to download it before it expires. * When the block content type is `IMAGE`: The content represents an image, consisting of: * `name`: The image name. * `url`: The S3 key or URL to the image. * `expires_at`: The expiration time for `url`. To save the image for future use, make sure to download it before it expires. * When the block content type is `SOURCES`: The content represents the source of the response block, including: * `id`: The ID of the source. * `content`: The chunk content. * `page_no`: The location of the chunk in the source file. * `source`: The file name of the data source. * `datasource_id`: The ID of the data source. * `dataset_id`: The ID of the dataset. * `file_type`: The name extension of the data source file. * When the block content type is `QUESTIONS`: The content represents follow-up questions suggested by Powerdrill. Let's see an example. For details about the `POST /v2/jobs` endpoint, see [Create job](create-job). ```python Python request theme={null} import requests url = "https://ai.data.cloud/api/v2/team/jobs" payload = { "session_id": "cxxdgegeegeg3433fff", "user_id": "tmm-dafasdfasdfasdf", "stream": True, "question": "Which travel agency has the highest average booking price?", "dataset_id": "cm1gjmg8e0057r3x22v1fdu8m", "datasource_ids": ["cm1gjmmoo0001h0x24uk1xgu9"], "output_language": "AUTO", "job_mode": "AUTO" } headers = { "x-pd-api-key": "", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) } response = requests.request("POST", url, headers=headers, data=payload) print(response.text) ``` The response is simlilar to this: ```` event:JOB_ID data:job-cm738w2ly00bb01l10mr4i2mx id:96ddde95-4885-4e71-9d0d-bcc1245c2e7f event:TASK data:{"id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","status":"running","parent_id":null,"stage":"Analyze","properties":{}}},"index":0,"finish_reason":null}],"created":1739445418,"group_id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","group_name":"Analyze","stage":"Analyze"} id:96ddde95-4885-4e71-9d0d-bcc1245c2e7f event:TASK data:{"id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445418,"group_id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","group_name":"Analyze","stage":"Analyze"} id:96ddde95-4885-4e71-9d0d-bcc1245c2e7f event:TASK data:{"id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","status":"done","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445418,"group_id":"96ddde95-4885-4e71-9d0d-bcc1245c2e7f","group_name":"Analyze","stage":"Analyze"} id:50c14384-2ae3-4351-93a3-3f43db55ef9e event:TASK data:{"id":"50c14384-2ae3-4351-93a3-3f43db55ef9e","model":"","choices":[{"delta":{"content":{"name":"Understand data","id":"50c14384-2ae3-4351-93a3-3f43db55ef9e","status":"running","parent_id":null,"stage":"Analyze","properties":{}}},"index":0,"finish_reason":null}],"created":1739445424,"group_id":"50c14384-2ae3-4351-93a3-3f43db55ef9e","group_name":"Understand data","stage":"Analyze"} id:3cd93b66-bf6f-4532-9ca4-e08119364c03 event:TASK data:{"id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","model":"","choices":[{"delta":{"content":{"name":"Calculate the average booking price for each travel agency. First, group by the travel agency, then calculate the average value of the price column for each agency.","id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":"junlan.csv"}}},"index":0,"finish_reason":null}],"created":1739445424,"group_id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","group_name":"Calculate the average booking price for each travel agency. First, group by the travel agency, then calculate the average of the price column for each agency.","stage":"Analyze"} id:3cd93b66-bf6f-4532-9ca4-e08119364c03 event:TASK data:{"id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","model":"","choices":[{"delta":{"content":{"name":"Calculate the average booking price for each travel agency. First, group by the travel agency, then calculate the average of the price column for each agency.","id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":"junlan.csv"}}},"index":0,"finish_reason":null}],"created":1739445424,"group_id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","group_name":"Calculate the average booking price for each travel agency. First, group by the travel agency, then calculate the average of the price column for each agency.","stage":"Analyze"} id:-1 event:TASK data:{"id":"-1","model":"","choices":[{"delta":{"content":{"name":"Search references","id":"4e6af71f-4e82-4e92-815f-5c1da6e94361","status":"running","parent_id":null,"stage":"Analyze","properties":{}}},"index":0,"finish_reason":null}],"created":1739445424,"group_id":"-1","group_name":"","stage":"Analyze"} id:3cd93b66-bf6f-4532-9ca4-e08119364c03 event:CODE data:{"id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","model":"","choices":[{"delta":{"role":null,"content":"```python\n\nimport pandas as pd\n\ndef invoke(input_0: pd.DataFrame) -> pd.DataFrame:\n '''\n input_0: pd.DataFrame junlan_table_0.csv\n '''\n # Group by travel agency and calculate the mean price\n result = input_0.groupby('travel_agency')['price'].mean().reset_index()\n # Rename columns for clarity\n result.columns = ['Travel Agency', 'Average Booking Price']\n return result\n\n# Assuming input_0 is the DataFrame provided\noutput = invoke(input_0)\n\n```"},"index":0,"finish_reason":null}],"created":1739445428,"group_id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","group_name":"Calculate the average booking price for each travel agency. First, group by travel agency, and then compute the average of the price column for each agency.","stage":"Analyze"} id:-1 event:TASK data:{"id":"-1","model":"","choices":[{"delta":{"content":{"name":"Search references","id":"4e6af71f-4e82-4e92-815f-5c1da6e94361","status":"done","parent_id":null,"stage":"Analyze","properties":{}}},"index":0,"finish_reason":null}],"created":1739445429,"group_id":"-1","group_name":"","stage":"Analyze"} id:50c14384-2ae3-4351-93a3-3f43db55ef9e event:TASK data:{"id":"50c14384-2ae3-4351-93a3-3f43db55ef9e","model":"","choices":[{"delta":{"content":{"name":"Understand data","id":"50c14384-2ae3-4351-93a3-3f43db55ef9e","status":"done","parent_id":null,"stage":"Analyze","properties":{}}},"index":0,"finish_reason":null}],"created":1739445429,"group_id":"50c14384-2ae3-4351-93a3-3f43db55ef9e","group_name":"Understand data","stage":"Analyze"} id:3cd93b66-bf6f-4532-9ca4-e08119364c03 event:TABLE data:{"id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","model":"","choices":[{"delta":{"content":{"name":"average_price_per_agency.csv","url":"https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/d65e5112-9615-4fcd-831b-fb9b7bf5da70.csv","expires_at":"2025-02-13T11:27:13.025667Z"}},"index":0,"finish_reason":null}],"created":1739445432,"group_id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","group_name":"Calculate the average booking price for each travel agency. First, group by travel agency, and then compute the average of the price column for each agency.","stage":"Analyze"} id:3cd93b66-bf6f-4532-9ca4-e08119364c03 event:TASK data:{"id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","model":"","choices":[{"delta":{"content":{"name":"Calculate the average booking price for each travel agency. First, group by travel agency, and then compute the average of the price column for each agency.","id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","status":"done","parent_id":null,"stage":"Analyze","properties":{"files":"junlan.csv"}}},"index":0,"finish_reason":null}],"created":1739445432,"group_id":"3cd93b66-bf6f-4532-9ca4-e08119364c03","group_name":"Calculate the average booking price for each travel agency. First, group by travel agency, and then compute the average of the price column for each agency.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":"average_price_per_agency.csv"}}},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"error","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":"average_price_per_agency.csv"}}},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:CODE data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"role":null,"content":"```python\n\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport io\n\ndef invoke(average_price_per_agency: pd.DataFrame) -> io.BytesIO:\n # Sort the data by 'Average Booking Price' in descending order\n sorted_data = average_price_per_agency.sort_values(by='Average Booking Price', ascending=False)\n \n # Create a bar plot\n plt.figure(figsize=(10, 6))\n plt.bar(sorted_data['Travel Agency'], sorted_data['Average Booking Price'])\n \n # Rotate x-axis labels for better readability\n plt.xticks(rotation=45, ha='right')\n \n # Add labels and title\n plt.xlabel('Travel Agency')\n plt.ylabel('Average Booking Price')\n plt.title('Average Booking Price per Travel Agency')\n \n # Adjust layout\n plt.tight_layout()\n \n # Save the plot to a BytesIO object\n output = io.BytesIO()\n plt.savefig(output, format='png')\n plt.close()\n \n # Seek to the beginning of the BytesIO object\n output.seek(0)\n \n return output\n\n```"},"index":0,"finish_reason":null}],"created":1739445433,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:IMAGE data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"url":"https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/ed2b799b-070a-4119-a3fa-04b8915583ce.png","name":"visualization.png","expires_at":"2025-02-13T11:27:15.594379Z"}},"index":0,"finish_reason":null}],"created":1739445435,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"done","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445435,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:af56c606-f813-4d4b-9d3e-76aff11fad85 event:TASK data:{"id":"af56c606-f813-4d4b-9d3e-76aff11fad85","model":"","choices":[{"delta":{"content":{"name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","id":"af56c606-f813-4d4b-9d3e-76aff11fad85","status":"done","parent_id":null,"stage":"Analyze","properties":{"files":""}}},"index":0,"finish_reason":null}],"created":1739445435,"group_id":"af56c606-f813-4d4b-9d3e-76aff11fad85","group_name":"Visualize the average booking price for each travel agency using a bar chart to easily compare which travel agency has the highest average price.","stage":"Analyze"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:TASK data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"content":{"name":"Conclusions","id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","status":"running","parent_id":null,"stage":"Respond","properties":{}}},"index":0,"finish_reason":null}],"created":1739445435,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"结论","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"\n\n`Analyzing Conclusions` \n\n"},"index":0,"finish_reason":null}],"created":1739445435,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"### Travel Agency with the Highest Average Booking Price\n\n"},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"#### Analysis\n"},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":""},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"\n"},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:TABLE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"content":{"name":"average_price_per_agency.csv","url":"https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/d65e5112-9615-4fcd-831b-fb9b7bf5da70.csv","expires_at":"2025-02-13T11:27:20.36399Z"}},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"\n\n"},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"- **Highest Average Booking Price**: Triprobotics Inc. has the highest average booking price of 279.04.\n\n"},"index":0,"finish_reason":null}],"created":1739445440,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"#### Visualization\n"},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":""},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"结论","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"\n"},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:IMAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"content":{"url":"https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/ed2b799b-070a-4119-a3fa-04b8915583ce.png","name":"visualization.png","expires_at":"2025-02-13T11:27:21.19331Z"}},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"\n\n"},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"- **Visual Confirmation**: The bar chart confirms that Triprobotics Inc. has the highest average booking price compared to other travel agencies.\n\n"},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"#### Conclusion and Insights\n"},"index":0,"finish_reason":null}],"created":1739445441,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:MESSAGE data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"role":null,"content":"- **Key Insight**: Triprobotics Inc. stands out with the highest average booking price among the listed travel agencies."},"index":0,"finish_reason":null}],"created":1739445442,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:QUESTIONS data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"content":["Analyze the distribution of booking prices across different travel agencies and identify any significant outliers or patterns.","Investigate the relationship between the trip type (oneway or roundtrip) and the average booking price for each travel agency.","Examine how the average booking price varies with different departure and arrival airports for the top three travel agencies with the highest average prices."]},"index":0,"finish_reason":null}],"created":1739445442,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:TRIGGER data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"content":{"name":"conclusion_slice","arguments":{"answer":"$answer"}}},"index":0,"finish_reason":null}],"created":1739445442,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:d7c2bdcc-e330-410c-ae01-b14fd4a6ca58 event:TASK data:{"id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","model":"","choices":[{"delta":{"content":{"name":"Conclusions","id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","status":"done","parent_id":null,"stage":"Respond","properties":{}}},"index":0,"finish_reason":null}],"created":1739445442,"group_id":"d7c2bdcc-e330-410c-ae01-b14fd4a6ca58","group_name":"Conclusions","stage":"Respond"} id:-1 event:SOURCES data:{"id":"-1","model":"","choices":[{"delta":{"content":[{"id":"1","source":"junlan.csv","page_no":null,"content":null,"datasource_id":"ds-cm5c11ati000301fcotg37qi4","dataset_id":"dset-cm5c11ao90bct01l1s07wxfjt","file_type":"csv","external_id":"11121434"}]},"index":0,"finish_reason":null}],"created":1739445442,"group_id":"-1","group_name":"","stage":"Analyze"} event:END_MARK data:[DONE] ```` However, if streaming is disabled, Powerdrill returns the response only after the entire response is ready. Let's use the same request as an example for a clear comparison. The only difference is that `stream` is set to `False`. ```python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/jobs" payload = { "session_id": "cxxdgegeegeg3433fff", "user_id": "tmm-dafasdfasdfasdf", "stream": False, "question": "Which travel agency has the highest average booking price?", "dataset_id": "cm1gjmg8e0057r3x22v1fdu8m", "datasource_ids": ["cm1gjmmoo0001h0x24uk1xgu9"], "output_language": "EN", "job_mode": "AUTO" } headers = { "x-pd-api-key": "$PROJECT_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` The response looks like this: ````json Example response theme={null} { { "code": 0, "msg": null, "data": { "job_id": "job-cm738e1ba004601l1rm95unn8", "blocks": [ { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(input_0: pd.DataFrame) -> pd.DataFrame:\n # Group by travel agency and calculate the mean price\n result = input_0.groupby('travel_agency')['price'].mean().reset_index()\n # Rename columns for clarity\n result.columns = ['Travel Agency', 'Average Booking Price']\n return result\n\n# Assuming input_0 is the DataFrame provided\noutput = invoke(input_0)\n\n```", "group_id": "e07b6a6a-7cf4-4fec-a068-c339cb1f3fc5", "group_name": "Calculate the average booking price for each travel agency. First, group by travel agency, and then calculate the average value of the price column for each agency.。", "stage": "Analyze" }, { "type": "TABLE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/f47e6d58-b0f1-405d-a367-d4055aab1188.csv", "name": "average_price_per_agency.csv", "expires_at": "2025-02-13T11:13:21.903308Z" }, "group_id": "e07b6a6a-7cf4-4fec-a068-c339cb1f3fc5", "group_name": "Calculate the average booking price for each travel agency. First, group by travel agency, then calculate the average value of the price column for each agency.", "stage": "Analyze" }, { "type": "CODE", "content": "```python\n\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport io\n\ndef invoke(average_price_per_agency: pd.DataFrame) -> io.BytesIO:\n # Sort the data by 'Average Booking Price' in descending order\n sorted_data = average_price_per_agency.sort_values(by='Average Booking Price', ascending=False)\n \n # Create a bar plot\n plt.figure(figsize=(10, 6))\n plt.bar(sorted_data['Travel Agency'], sorted_data['Average Booking Price'])\n \n # Rotate x-axis labels for better readability\n plt.xticks(rotation=45, ha='right')\n \n # Add labels and title\n plt.xlabel('Travel Agency')\n plt.ylabel('Average Booking Price')\n plt.title('Average Booking Price per Travel Agency')\n \n # Adjust layout\n plt.tight_layout()\n \n # Save the plot to a BytesIO object\n output = io.BytesIO()\n plt.savefig(output, format='png')\n plt.close()\n \n # Seek to the beginning of the BytesIO object\n output.seek(0)\n \n return output\n\n```", "group_id": "8580f207-1b66-4231-9266-69c845f2d2cd", "group_name": "Visualize the average booking price for each travel agency using a bar chart, so you can easily compare which agency has the highest average price.", "stage": "Analyze" }, { "type": "IMAGE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/b185c0f3-a021-4ceb-b317-b391c0e7df28.png", "name": "visualization.png", "expires_at": "2025-02-13T11:13:21.903308Z" }, "group_id": "8580f207-1b66-4231-9266-69c845f2d2cd", "group_name": "Use a bar chart to visualize the average booking price for each travel agency, so you can easily compare which agency has the highest average price.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "\n\n`Analyzing Conclusions` \n\n### Travel Agency with the Highest Average Booking Price\n\n#### Analysis\n\n", "group_id": "60ad92c9-6151-41a3-b3aa-ab695cdaa5f5", "group_name": "Conclusions", "stage": "Respond" }, { "type": "TABLE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/f47e6d58-b0f1-405d-a367-d4055aab1188.csv", "name": "average_price_per_agency.csv", "expires_at": "2025-02-13T11:13:21.903308Z" }, "group_id": "60ad92c9-6151-41a3-b3aa-ab695cdaa5f5", "group_name": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Highest Average Booking Price**: Triprobotics Inc. has the highest average booking price at **279.04**.\n\n#### Visualization\n\n", "group_id": "60ad92c9-6151-41a3-b3aa-ab695cdaa5f5", "group_name": "Conclusions", "stage": "Respond" }, { "type": "IMAGE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/tmm-cm5ao3yoe00zm01l1u1e7p3pj/b185c0f3-a021-4ceb-b317-b391c0e7df28.png", "name": "visualization.png", "expires_at": "2025-02-13T11:13:21.903308Z" }, "group_id": "60ad92c9-6151-41a3-b3aa-ab695cdaa5f5", "group_name": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Visual Confirmation**: The bar chart confirms that Triprobotics Inc. leads with the highest average booking price compared to other agencies.\n\n#### Conclusion and Insights\n- **Key Insight**: Triprobotics Inc. stands out with the highest average booking price among the listed travel agencies.", "group_id": "60ad92c9-6151-41a3-b3aa-ab695cdaa5f5", "group_name": "Conclusions", "stage": "Respond" }, { "type": "SOURCES", "content": [ { "source": "junlan.csv", "datasource_id": "ds-cm5c11ati000301fcotg37qi4", "dataset_id": "dset-cm5c11ao90bct01l1s07wxfjt", "file_type": "csv", "external_id": "11121434" } ], "group_id": "", "group_name": "", "stage": "Respond" }, { "type": "QUESTIONS", "content": [ "Analyze the distribution of booking prices across different travel agencies and identify any significant outliers or patterns.", "Investigate the relationship between the trip type (oneway or roundtrip) and the average booking price for each travel agency.", "Examine how the average booking price varies with different departure and arrival airports for the top three travel agencies with the highest average prices." ], "group_id": "-1", "group_name": null, "stage": "Respond" } ] } } ```` *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Upload local file Source: https://docs.powerdrill.ai/api-reference/v2/upload-file post /v2/team/file/upload-datasource Uploads a file. ### Introduction You can use this endpoint to upload your local file and then use the `file_key` you obtained to create a data source through the [Create data source](/api-reference/v2/create-data-source) endpoint. Only files with the following extensions are supported: **.csv**, **.tsv**, **.md**, **.mdx**, **.json**, **.txt**, **.pdf**, **.pptx**, **.ppt**, **.doc**, **.docx**, **.xls**, or **.xlsx**. For optimal upload efficiency, we recommend using this endpoint for files **smaller than 1 MB**. For files larger than 1 MB, please initiate a [multipart upload](/api-reference/v2/initiate-multipart-upload) task. **Example request:** ```curl cURL theme={null} curl --location 'http://ai.data.cloud/api/v2/team/file/upload-datasource' \ --header 'x-pd-api-key: ' \ --header 'x-pd-external-trace-id;' \ --header 'Cookie: Cookie_1=value; metabase.DEVICE=f3cb3dbb-ef62-4026-845e-1480431c751d' \ --form 'file=@"/Users/username/Downloads/c5168946-8f1a-4c92-a167-5e3be57745be.csv"' \ --form 'user_id="tmm-cm5ao3yoe00zm01l1u1e7p3pj"' ``` ```python Python theme={null} import requests url = "http://ai.data.cloud/api/v2/team/file/upload-datasource" payload = {'user_id': 'tmm-cm5ao3yoe00zm01l1u1e7p3pj'} files=[ ('file',('c5168946-8f1a-4c92-a167-5e3be57745be.csv',open('/Users/jiaoqi/Downloads/c5168946-8f1a-4c92-a167-5e3be57745be.csv','rb'),'text/csv')) ] headers = { 'x-pd-api-key': '', 'x-pd-external-trace-id': '', 'Cookie': 'Cookie_1=value; metabase.DEVICE=f3cb3dbb-ef62-4026-845e-1480431c751d' } response = requests.request("POST", url, headers=headers, data=payload, files=files) print(response.text) ``` **Example response:** ```json 200 theme={null} { "code": 0, "data": { "file_object_key": "/tmp/sdgsagdsgsadgasdg.csv" } } ``` # Excel AI Analysis Best Practices Source: https://docs.powerdrill.ai/best-practices/excel-ai-analysis-best-practices Best practices for using Powerdrill AI to analyze Excel files ## Proper data formatting Well-organized and clean data allows Powerdrill to carry out precise, error-free analysis and visualization. To ensure this, follow these best practices: ✅ **Column headers**: Use clear, descriptive headers in the first row. ✅ **Tabular structure**: Structure your data with records listed in rows beneath the headers. ✅ **Universality**: Use plain, widely understood language for column names, avoiding industry or company-specific terms. ### Some examples **Figure 3**: The spreadsheet is structured **Figure 4**: The main body of the spreadsheet is structured, though it includes titles and footnotes *** ## Improper data formatting Composite spreadsheets or poorly formatted data may prevent from analyzing or transforming the information. To avoid issues, be mindful of the following common pitfalls: ❌ **Multi-level headers** ❌ **Too many empty rows / columns** ❌ **Irregular tables contained** ❌ **Multiple sections** ### Some examples **Figure 1**: Spreadsheets with multi-level headers are difficult to index **Figure 2**: Spreadsheets with multiple sections, such as financial statements, should be formatted into tabular data # Quick Start Source: https://docs.powerdrill.ai/developer-guides/quick-start Start running your first data job on Powerdrill Enterprise Follow this guide to learn how to create a dataset, add data sources to the dataset, create a session associated with the dataset, and then run jobs to start anayzing the uploaded data. *** ## Step 1. Get your project API key If you're the admin of your team, [get API key of the target project](/enterprise/projects#manage-project-api-keys) on the admin console. If you're a system user or virtual user in a team, simply ask your admin to provide you with one. *** ## Step 2. Create a dataset and upload a data source to it This step is optional but highly recommended, as it allows you to receive insights tailored to your own data. Data sources are the data you upload to Powerdrill for embedding, indexing, knowledge extraction, and vectorized storage and retrieval, while datasets are collections of data sources that help organize and categorize them. You can create datasets and data sources in two ways: * **Method 1**: Create a dataset first, then add data sources to it. * **Method 2**: Create a data source directly without specifying a dataset, and Powerdrill will automatically create a dataset for it. 1. Make a request to `POST /v1/team/datasets` endpoint to create a dataset. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v1/team/datasets \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "name": "My dataset", "description": "my default dataset", "userId": "tmm-cm5m7khoz52zh07n4m7x1ut60" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v1/team/datasets" payload = { "name": "My dataset", "description": "my default dataset", "userId": "tmm-cm5m7khoz52zh07n4m7x1ut60" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key). **Example response**: ```json theme={null} { "code": 0, "data": { "id": "cm3my37en3q36017q7x3hyyf4" } } ``` Obtain the `id` value (dataset ID) from the response and save it for later use. 2. Make a request to the [`POST /v1/team/datasets/{datasetId}/datasources`](create-data-source) endpoint. Replace the `datasetId` value with the ID of the dataset you've created in the previous sub-step When making the request, specify either `url` or `fileKey`, **but not both**. Use `url` to upload a file through a publicly accessible URL. For privately accessible files, use `fileKey`. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v1/team/datasets/{datasetId}/datasources \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "name": "test.csv", "fileName": "test.csv", "type": "FILE", "url": "https://s3.amazonaws.com/powerdrilltest/user/clvl4cad2001q01l1m522hxlu/upload/f9773f1e-cd68-489a-8121-d566ca9218b1.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240924T143419Z&X-Amz-SignedHeaders=host&X-Amz-Expires=599&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20240924%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=9ca0c58d508926a5811818041d557ffb53c64025dae94c0855280d457c7089a2", "userId": "tmm-cm5m7khoz52zh07n4m7x1ut60" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v1/team/datasets/{datasetId}/datasources" payload = { "name": "test.csv", "fileName": "test.csv", "type": "FILE", "url": "https://s3.amazonaws.com/powerdrilltest/user/clvl4cad2001q01l1m522hxlu/upload/f9773f1e-cd68-489a-8121-d566ca9218b1.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240924T143419Z&X-Amz-SignedHeaders=host&X-Amz-Expires=599&X-Amz-Credential=AKIARLSQLXURHEIDN4OZ%2F20240924%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=9ca0c58d508926a5811818041d557ffb53c64025dae94c0855280d457c7089a2", "userId": "tmm-cm5m7khoz52zh07n4m7x1ut60" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` **Example response**: ```json theme={null} { "code": 0, "data": { "id": "cm3myfsfc03jn011csb8wah6p", "datasetId": "cm3my37en3q36017q7x3hyyf4", "name": "test.pdf", "fileName": "test.pdf", "type": "FILE", "status": "pending" } } ``` Repeat this sub-step to create multiple data sources in the same dataset. Make a request to the [POST /v1/team/datasources](create-data-source-without-dataset) endpoint. **Example request**: ```curl cURL theme={null} curl --location 'https://ai.data.cloud/api/v1/team/datasets/cm3my37en3q36017q7x3hyyf4/datasources' \ --header 'x-pd-api-key: $PD_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "name": "test.pdf", "fileName": "test.pdf", "type": "FILE", "url": "https://arxiv.org/pdf/2406.12660v1.pdf" }' ``` ```python Python theme={null} import requests import json url = "https://ai.data.cloud/api/v1/team/datasets/cm3my37en3q36017q7x3hyyf4/datasources" payload = json.dumps({ "name": "test.pdf", "fileName": "test.pdf", "type": "FILE", "url": "https://arxiv.org/pdf/2406.12660v1.pdf" }) headers = { 'x-pd-api-key': '$PD_API_KEY', 'Content-Type': 'application/json' } response = requests.request("POST", url, headers=headers, data=payload) print(response.text) ``` Specify either `url` or `fileKey`, but not both. Use `url` to upload a file through a publicly accessible URL. For privately accessible files, use `fileKey` (this feature will be supported soon). **Example response**: ```json theme={null} { "code": 0, "data": { "id": "cm3myfsfc03jn011csb8wah6p", "datasetId": "cm3my37en3q36017q7x3hyyf4", "name": "test.pdf", "fileName": "test.pdf", "type": "FILE", "status": "pending" } } ``` Obtain the `datasetId` value (dataset ID) from the response and save it for later use. ## Step 3. Create a session To create a session, make a request to the [POST /v1/team/sessions](create-session) endpoint. Sessions are essential for running jobs on Powerdrill, as each job must be linked to a session using its session ID. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v1/team/sessions \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "title": "My session", "languageType": "AUTO", "jobMode": "AUTO", "maxMessagesInContext": 10, "userId": "tmm-egejgowqg=g=egen" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v1/team/sessions" payload = { "title": "My session", "languageType": "AUTO", "jobMode": "AUTO", "maxMessagesInContext": 10, "userId": "tmm-egejgowqg=g=egen" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` When making a request: * Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key). * Set the user ID to your actual user ID. **Example response**: ```json theme={null} { "code": 0, "data": { "id": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2" } } ``` Obtain the `id` value (session ID) from the response and save it for use in the following step. *** ## Step 4. Create a job Now, after you've prepared a session and probably a dataset stuffed with data sources, you can create a job to start conversing with Powerdrill. For the definition of job, see [What Is Job?](/enterprise/what-is-job). Make a request to the [POST /v1/team/jobs](create-job) endpoint. Powerdrill provides the ability to stream responses, controlled by the `stream` parameter. For more details about how to understand the streaming mode, see [Streaming](streaming). * If `stream` is set to true, streaming is enabled. * If `stream` is set to false, streaming is disabled. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v1/team/jobs \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "datasetId": "cm1gjmg8e0057r3x22v1fdu8m", "datasourceIdList": [ "cm1gjmmoo0001h0x24uk1xgu9" ], "languageType": "EN", "question": "Hello World", "sessionId": "5534f591-1520-4e74-b753-b87615b2c57a", "stream": false }' ``` ```python Python theme={null} import requests import json url = "https://ai.data.cloud/api/v1/team/jobs" payload = json.dumps({ "datasetId": "cm3my37en3q36017q7x3hyyf4", "datasourceIdList": [ "cm3myfsfc03jn011csb8wah6p" ], "languageType": "EN", "question": "How do 'Digital Services for Taxpayers' scores vary across economies, and which economy has the most advanced digital services?", "sessionId": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2", "stream": True }) headers = { 'x-pd-api-key': '$PD_API_KEY', 'x-pd-api-agent-id': 'GENERAL', 'Content-Type': 'application/json' } response = requests.request("POST", url, headers=headers, data=payload) print(response.text) ```
``` event:JOB_ID data:job-cm3ik4yhz01vg01l17yk4467y event:TITLE_GENERATION data:test session id:2b5cba4a-d8a5-4beb-aef9-a8612828c405 event:TASK data:{"id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","status":"running","stage":"Analyze","properties":{}}},"index":0}],"created":1731664172,"groupId":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","groupName":"Analyze","stage":"Analyze"} id:2b5cba4a-d8a5-4beb-aef9-a8612828c405 event:TASK data:{"id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","status":"running","stage":"Analyze","properties":{"files":""}}},"index":0}],"created":1731664172,"groupId":"2b5cba4a-d8a5-4beb-aef9-a8612828c405","groupName":"Analyze","stage":"Analyze"} ... ... ... id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:IMAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":{"url":"https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/5e3a766c-8d16-4fc0-b06f-7a337196186d.png","name":"Trends of Deaths from Different Disaster Types Over the Years","expiredAt":"2024-11-21T09:50:00.026476Z"}},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"\n\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Chart Description**: The chart illustrates the trend of death tolls over time for various types of disasters.\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Key Trends**:\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":" - Droughts and floods had high death tolls in the early 20th century.\n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":" - Droughts and floods had higher death tolls in the early 20th century. \n"},"index":0}],"created":1731664200,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":" - Death tolls from other disaster types remained relatively low and stable. \n\n"},"index":0}],"created":1731664201,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"#### Conclusions and Insights\n"},"index":0}],"created":1731664201,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Significant Impact of Droughts and Floods**: In the early 20th century, droughts and floods caused significantly higher death tolls compared to other disasters.\n"},"index":0}],"created":1731664201,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:MESSAGE data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":"- **Fluctuations in Earthquakes and Extreme Weather**: These disasters had significant impacts on death tolls in certain years, highlighting the need for enhanced preventive measures."},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:QUESTIONS data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":["Please analyze the distribution of deaths caused by different types of natural disasters across various countries and regions, and generate a corresponding ECharts map chart.","Please compare the changes in death tolls caused by natural disasters across different decades, and generate a corresponding ECharts line chart.","Please analyze the global death toll trend of a specific natural disaster (e.g., earthquakes) and generate a corresponding ECharts bar chart."]},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:TRIGGER data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":{"name":"conclusion_slice","arguments":{"answer":"$answer"}}},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3 event:TASK data:{"id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","model":"","choices":[{"delta":{"content":{"name":"Conclusions","id":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","status":"done","stage":"Respond","properties":{}}},"index":0}],"created":1731664202,"groupId":"ccd0c9a2-b8d5-4066-8d29-2f54c87a60a3","groupName":"Conclusions","stage":"Respond"} id:-1 event:SOURCES data:{"id":"-1","model":"","choices":[{"delta":{"content":[{"id":"1","source":"makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv","datasourceId":"clxin6l9200oo01l1457bolx3","datasetId":"clxin6l8400ok01l1ff2m0s25","fileType":"csv","externalId":"clxin6l0h001901hzxhjaae6q"}]},"index":0}],"created":1731664202,"groupId":"-1","groupName":"","stage":"Analyze"} event:END_MARK data:[DONE] ```
**Example request**: ```curl cURL theme={null} curl --location 'https://ai.data.cloud/api/v1/team/jobs' \ --header 'x-pd-api-key: $PD_API_KEY' \ --header 'x-pd-api-agent-id: GENERAL' \ --header 'Content-Type: application/json' \ --data '{ "datasetId": "cm3my37en3q36017q7x3hyyf4", "datasourceIdList": [ "cm3myfsfc03jn011csb8wah6p" ], "languageType": "EN", "question": "How do '\''Digital Services for Taxpayers'\'' scores vary across economies, and which economy has the most advanced digital services?", "sessionId": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2", "stream": false }' ``` ```python Python theme={null} import requests import json url = "https://ai.data.cloud/api/v1/team/jobs" payload = json.dumps({ "datasetId": "cm3my37en3q36017q7x3hyyf4", "datasourceIdList": [ "cm3myfsfc03jn011csb8wah6p" ], "languageType": "EN", "question": "How do 'Digital Services for Taxpayers' scores vary across economies, and which economy has the most advanced digital services?", "sessionId": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2", "stream": False }) headers = { 'x-pd-api-key': '$PD_API_KEY', 'x-pd-api-agent-id': 'GENERAL', 'Content-Type': 'application/json' } response = requests.request("POST", url, headers=headers, data=payload) print(response.text) ``` ```node.js Nodejs theme={null} var request = require('request'); var options = { 'method': 'POST', 'url': 'https://ai.data.cloud/api/v1/team/jobs', 'headers': { 'x-pd-api-key': '$PD_API_KEY', 'x-pd-api-agent-id': 'GENERAL', 'Content-Type': 'application/json' }, body: JSON.stringify({ "datasetId": "cm3my37en3q36017q7x3hyyf4", "datasourceIdList": [ "cm3myfsfc03jn011csb8wah6p" ], "languageType": "EN", "question": "How do 'Digital Services for Taxpayers' scores vary across economies, and which economy has the most advanced digital services?", "sessionId": "4440ab38-3df0-465b-a66c-bf6acb0f1bc2", "stream": false }) }; request(options, function (error, response) { if (error) throw new Error(error); console.log(response.body); }); ```
````json theme={null} { "code": 0, "data": { "jobId": "job-cm3ikdeuj02zk01l1yeuirt77", "blocks": [ { "type": "CODE", "content": "```python\n\nimport pandas as pd\n\ndef invoke(input_0: pd.DataFrame) -> pd.DataFrame:\n '''\n input_0: pd.DataFrame makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv\n '''\n # Group by 'Year' and sum the deaths for each type of disaster\n aggregated_data = input_0.groupby('Year').sum().reset_index()\n \n # Select only the columns related to deaths\n death_columns = [\n 'Deaths - Drought (decadal)', 'Deaths - Flood (decadal)', \n 'Deaths - Earthquake (decadal)', 'Deaths - Extreme weather (decadal)', \n 'Deaths - Extreme temperature (decadal)', 'Deaths - Volcanic activity (decadal)', \n 'Deaths - Wildfire (decadal)', 'Deaths - Glacial lake outburst flood (decadal)', \n 'Deaths - Dry mass movement (decadal)', 'Deaths - Wet mass movement (decadal)', \n 'Deaths - Fog (decadal)'\n ]\n \n # Create a new DataFrame with the aggregated results\n output = aggregated_data[['Year'] + death_columns]\n \n # Rename columns to be more descriptive\n output.columns = ['Decade'] + [col.replace('Deaths - ', '').replace(' (decadal)', '') for col in death_columns]\n \n return output\n\n```", "groupId": "33063572-6e88-4912-8e2d-4166bcc8caee", "groupName": "Analyze the dataset to observe the trend of deaths caused by different types of natural disasters over the past century. This involves aggregating the data by decade and calculating the total number of deaths for each type of disaster to identify any changes in trends.", "stage": "Analyze" }, { "type": "TABLE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/e24b6a5f-fdb8-48ca-ae35-dc91ac8e8ef7.csv", "name": "trend_data.csv", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "33063572-6e88-4912-8e2d-4166bcc8caee", "groupName": "Analyze the dataset to observe the trend of deaths caused by different types of natural disasters over the past century. This involves aggregating the data by decade and calculating the total number of deaths for each type of disaster to identify any changes in trends.", "stage": "Analyze" }, { "type": "IMAGE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/81b75a33-a223-4954-9680-9f397872c8ad.png", "name": "Trend of Deaths from Natural Disasters Over the Century", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "7501680b-5879-441b-bd96-f58b1029ae17", "groupName": "Visualize the trend data to show how the number of deaths from different types of natural disasters has changed over the past century. Use line charts to represent the trends for each disaster type, which will help in understanding the impact of measures and technological advancements on reducing deaths.", "stage": "Analyze" }, { "type": "MESSAGE", "content": "\n\n`Analyzing Conclusions` \n\n### Analysis of Trends in Natural Disaster Fatalities\n\n#### Data Analysis\n\n", "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "TABLE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/e24b6a5f-fdb8-48ca-ae35-dc91ac8e8ef7.csv", "name": "trend_data.csv", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Droughts and Floods**: In the early 20th century, droughts and floods caused extremely high death tolls, particularly in the 1920s and 1930s.\n- **Earthquakes and Extreme Weather**: Earthquakes and extreme weather also led to significant fatalities throughout the century, especially in the 1970s and 1990s.\n- **Extreme Temperatures and Volcanic Activity**: These disasters had relatively lower death tolls, but in certain decades, such as the 2000s, deaths caused by extreme temperatures increased.\n\n#### Trend Visualization\n\n", "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "IMAGE", "content": { "url": "https://static.powerdrill.ai/tmp_datasource_cache/code_result/cm37bchx106e301l1v9yf67yc/81b75a33-a223-4954-9680-9f397872c8ad.png", "name": "Trend of Deaths from Natural Disasters Over the Century", "expiredAt": "2024-11-21T09:56:34.290544Z" }, "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "MESSAGE", "content": "\n\n- **Overall Trend**: The chart shows that, despite spikes in death tolls caused by natural disasters in certain decades, the overall trend is declining.\n- **Impact of Technology and Measures**: Over time, advancements in technology and the strengthening of disaster prevention measures are likely key factors contributing to the reduction in fatalities.\n\n#### Conclusions and Insights\n- **Technological Advancements**: Progress in modern technology, such as improved early warning systems and construction techniques, may have reduced deaths caused by earthquakes and extreme weather.\n- **Disaster Prevention Measures**: The enhancement of global disaster prevention efforts and emergency response capabilities has likely played a crucial role in mitigating the fatality rates of natural disasters.", "groupId": "b842aca7-6fd5-4190-85fa-97085e473877", "groupName": "Conclusions", "stage": "Respond" }, { "type": "SOURCES", "content": [ { "source": "makeovermonday-a-century-of-global-deaths-from-disasters_decadal-deaths-disasters-type.csv", "datasourceId": "clxin6l9200oo01l1457bolx3", "datasetId": "clxin6l8400ok01l1ff2m0s25", "fileType": "csv", "externalId": "clxin6l0h001901hzxhjaae6q" } ], "groupId": "", "groupName": "", "stage": "Respond" }, { "type": "QUESTIONS", "content": [ "Analyze the changes in death toll trends for different types of natural disasters over the past century and explore which types of disasters have experienced the most significant reductions in fatalities.", "Study the differences in technological advancements and measures for responding to natural disasters across various regions globally, and analyze how these differences have influenced changes in death tolls in each region.", "Explore how potential future technological advancements and policy measures could further reduce fatalities caused by natural disasters, and assess their feasibility and potential impacts." ], "groupId": "-1", "stage": "Respond" } ] } } ````
When making a request: * Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-api-key). * Since this topic covers running a general job and no data agent is used, set the `x-pd-api-agent-id` header to `GENERAL` (uppercase). * Replace the `sessionId` value with the ID of the session you've created in [Step 3](#step-3-create-a-session). * To enable Powerdrill to retrieve information from your own data and provide responses specific to it, set the `datasetId` to the ID of the dataset obtained in [Step 2](#step-2-create-a-dataset-and-a-data-source). # Quick Start Source: https://docs.powerdrill.ai/developer-guides/quick-start-v2 Start running your first data job on Powerdrill Enterprise Follow this guide to learn how to create a dataset, add data sources to the dataset, create a session associated with the dataset, and then run jobs to start anayzing the uploaded data. *** ## Step 1. Get your project API key If you're the admin of your team, [get API key of the target project](/enterprise/projects#manage-project-api-keys) on the admin console. If you're a system user or virtual user in a team, simply ask your admin to provide you with one. *** ## Step 2. Create a dataset and upload a data source to it This step is optional but highly recommended, as it allows you to receive insights tailored to your own data. Data sources are the data you upload to Powerdrill for embedding, indexing, knowledge extraction, and vectorized storage and retrieval, while datasets are collections of data sources that help organize and categorize them. You can create datasets and data sources in two ways: * **Method 1**: Create a dataset first, then add data sources to it. * **Method 2**: Create a data source directly without specifying a dataset, and Powerdrill will automatically create a dataset for it. 1. Make a request to [POST /v2/team/datasets](/api-reference/v2/create-dataset) endpoint to create a dataset. Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key) and `$UID` with your user ID in the target project. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v2/team/datasets \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "name": "My dataset", "description": "my default dataset", "user_id": "$UID" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets" payload = { "name": "My dataset", "description": "my default dataset", "user_id": "$UID" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` **Example response**: ```json theme={null} { "code": 0, "data": { "id": "dset-cmc1nh2e2lqf507retfodc0dn" } } ``` Obtain the `id` value (dataset ID) from the response and save it for later use. 2. Make a request to the [`POST /v2/team/datasets/{id}/datasources`](/api-reference/v2/create-data-source) endpoint. Replace the `id` value with the ID of the dataset you've created in the previous sub-step. Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key) and `$UID` with your user ID in the target project. When making the request, specify either `url` or `file_key`, **but not both**. Use `url` to upload a file through a publicly accessible URL. For privately accessible files, use `file_key`. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v2/team/datasets/dset-cmc1nh2e2lqf507retfodc0dn/datasources \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "name": "test.pdf", "type": "FILE", "user_id": "$UID", "url": "https://arxiv.org/pdf/2406.12660v1" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets/dset-cmc1nh2e2lqf507retfodc0dn/datasources" payload = { "name": "test.pdf", "type": "FILE", "user_id": "$UID", "url": "https://arxiv.org/pdf/2406.12660v1" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` **Example response**: ```json theme={null} { "code": 0, "data": { "id": "ds-cmc1ntbw105ug07j49zei4kcb", "name": "test.pdf", "type": "FILE", "status": "synching", "dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn" } } ``` Repeat this sub-step to create multiple data sources in the same dataset, if needed. Make a request to the [POST /v2/team/datasources](/api-reference/create-data-source-without-dataset) endpoint. Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key) and `$UID` with your user ID in the target project. Specify either `url` or `file_key`, but not both. Use `url` to upload a file through a publicly accessible URL. For privately accessible files, use `file_key` (this feature will be supported soon). **Example request**: ```curl cURL theme={null} curl --request GET \ --url "https://ai.data.cloud/api/v2/team/datasets/dset-cmc1nh2e2lqf507retfodc0dn/datasources/ds-cmc1ntbw105ug07j49zei4kcb?user_id=$UID" \ --header 'x-pd-api-key: $PD_API_KEY' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/datasets" payload = { "name": "My dataset", "description": "my default dataset", "user_id": "$UID" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` **Example response**: ```json theme={null} { "code": 0, "data": { "id": "ds-cmc1ntbw105ug07j49zei4kcb", "name": "test.pdf", "type": "FILE", "status": "synched", "size":781886, "dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn" } } ``` Obtain the `dataset_id` value (dataset ID) from the response and save it for later use. *** ## Step 3. Create a session To create a session, make a request to the [POST /v2/team/sessions](/api-reference/create-session) endpoint. Sessions are essential for running jobs on Powerdrill, as each job must be linked to a session using its session ID. Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key) and `$UID` with your user ID in the target project. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v2/team/sessions \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "name": "My session", "output_language": "EN", "job_mode": "AUTO", "max_contextual_job_history": 10, "user_id": "$UID" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/sessions" payload = { "name": "My session1", "output_language": "FR", "job_mode": "AUTO", "max_contextual_job_history": 10, "user_id": "$UID" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ``` **Example response**: ```json theme={null} { "code": 0, "data": { "id": "bc9a8127-4214-42b2-bbbe-a022f23d9795" } } ``` Obtain the `id` value (session ID) from the response and save it for use in the following step. *** ## Step 4. Create a job Now, after you've prepared a session and probably a dataset stuffed with data sources, you can create a job to start conversing with Powerdrill. For the definition of job, see [What Is Job?](/enterprise/what-is-job). Make a request to the [POST /v2/team/jobs](/api-reference/v2/create-job) endpoint. Powerdrill provides the ability to stream responses, controlled by the `stream` parameter. For more details about how to understand the streaming mode, see [Streaming](streaming). * If `stream` is set to true, streaming is enabled. * If `stream` is set to false, streaming is disabled. When making the request: * Replace `$PD_API_KEY` with the API key you've obtained in [Step 1](#step-1-get-your-project-api-key) and `$UID` with your user ID in the target project. * Replace the `session_id` value with the ID of the session you've created in [Step 3](#step-3-create-a-session). * To enable Powerdrill to retrieve information from your own data and provide responses specific to it, set the `dataset_id` to the ID of the dataset obtained in [Step 2](#step-2-create-a-dataset-and-a-data-source). * If you want to use specific data sources in the specified dataset, list the data source IDs in the `datasource_ids` field. **Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v2/team/jobs \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "session_id": "bc9a8127-4214-42b2-bbbe-a022f23d9795", "user_id": "$UID", "stream": true, "question": "introducing the dataset", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "datasource_ids": [ "ds-cmc1ntbw105ug07j49zei4kcb" ], "output_language": "EN", "job_mode": "AUTO" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/jobs" payload = { "session_id": "bc9a8127-4214-42b2-bbbe-a022f23d9795", "user_id": "$UID", "stream": True, "question": "introducing the dataset", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "datasource_ids": ["ds-cmc1ntbw105ug07j49zei4kcb"], "output_language": "EN", "job_mode": "AUTO" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ```
``` event:JOB_ID data:job-cmc1ohgsllys907reaaowv4fl id:9fbe40de-5c5b-4e27-b82e-ecf727b15248 event:TASK data:{"id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","status":"running","parent_id":null,"stage":"Analyze","properties":{}}},"finish_reason":null,"index":0}],"created":1750234590,"group_id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","group_name":"Analyze","stage":"Analyze"} :keep-alive id:9fbe40de-5c5b-4e27-b82e-ecf727b15248 event:TASK data:{"id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","status":"running","parent_id":null,"stage":"Analyze","properties":{"files":"test.pdf"}}},"finish_reason":null,"index":0}],"created":1750234590,"group_id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","group_name":"Analyze","stage":"Analyze"} id:9fbe40de-5c5b-4e27-b82e-ecf727b15248 event:TASK data:{"id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","model":"","choices":[{"delta":{"content":{"name":"Analyze","id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","status":"done","parent_id":null,"stage":"Analyze","properties":{"files":"test.pdf"}}},"finish_reason":null,"index":0}],"created":1750234591,"group_id":"9fbe40de-5c5b-4e27-b82e-ecf727b15248","group_name":"Analyze","stage":"Analyze"} id:f5e680eb-3762-481b-826f-483a8e74e268 event:TASK data:{"id":"f5e680eb-3762-481b-826f-483a8e74e268","model":"","choices":[{"delta":{"content":{"name":"Search summary","id":"f5e680eb-3762-481b-826f-483a8e74e268","status":"running","parent_id":null,"stage":"Analyze","properties":{}}},"finish_reason":null,"index":0}],"created":1750234592,"group_id":"f5e680eb-3762-481b-826f-483a8e74e268","group_name":"Search summary","stage":"Analyze"} id:f5e680eb-3762-481b-826f-483a8e74e268 event:SOURCES data:{"id":"f5e680eb-3762-481b-826f-483a8e74e268","model":"","choices":[{"delta":{"content":[{"id":"1","source":"test.pdf","page_no":null,"content":"summary: test.pdf\nAn experiment with 562 participants investigated the impact of Explainable AI (XAI) and AI literacy on user compliance. Results revealed that XAI boosts compliance, influenced by AI literacy, with the relationship mediated by users' mental model of AI. This study highlights the importance of XAI in AI-based system design. It explores the connection between AI literacy, mental models, XAI techniques, and user compliance with AI recommendations. The research also examines the effect of presenting different XAI types on user compliance. An AI artifact was developed to predict age from photographs, offering personalized explanations to enhance decision-making and compliance with AI recommendations. The study delves into AI interpretability, AI literacy, explainable AI models, and their influence on user behavior. It discusses advancements in AI, machine learning, and user interaction, addressing areas like facial recognition, digital resilience, and algorithmic fairness.","datasource_id":"ds-cmc1ntbw105ug07j49zei4kcb","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null},{"id":"2","source":"test.pdf","page_no":null,"content":"summary: test.pdf\nAn experiment with 562 participants investigated the impact of Explainable AI (XAI) and AI literacy on user compliance. Results revealed that XAI boosts compliance, influenced by AI literacy, with the relationship mediated by users' mental model of AI. This study highlights the importance of designing AI systems with XAI for better user engagement.","datasource_id":"ds-cmc1nrv4a05ue07j4vscij2z7","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null},{"id":"3","source":"test.pdf","page_no":null,"content":"7 \nThe decision for a data set for building an AI for age estimation is tightly bound to the current research basis on ML \nmodels for age estimation. Age estimation has been of particular interest in the ML community, and many researchers have \ntackled the task of predicting the age of a person on an image [11,47]. The largest and most popular data set is the IMDB-\nWIKI data set [47], which we utilize for training our AI. For our implementation, we take advantage of the source code \npublished by Serengil [51], with minor adjustments in Python, using the popular keras package. The model itself is based \non a CNN, which uses the VGG-16 architecture and is pre-trained on the FaceNet database [50]. The network architecture \nis then adjusted to the age estimation task and our specific data set. \nWhile the IMDB-WIKI data set is widely used as a training basis for age estimation models and the use of existing, \npublished models makes them convenient to use, there are multiple reasons for which the pictures in this data set cannot \nbe used for display (≠ model training) in our study; the quality of the images varies vastly, the ages of the persons are not \nvalidated, and the data set contains many pictures of celebrities. Especially the latter could falsify the participants' \nperformance as they might have existing knowledge of the age of a person. Another factor that might have an unintended \neffect on the study is the fact that the images are taken “in the wild”, meaning that there is no standard way of how the \npeople are shown in the image. The people are pictured in various ways, with different poses, facial expressions like smiles \nor laughter, and clothing like sunglasses, headgear, or jewelry. To address these shortcomings, we use the MORPH data \nset Feld for model adoption and presentation to the study participants[46]. It has been specifically developed for research \npurposes and contains the actual age of the people depicted in the pictures. While there are multiple versions of MORPH, \nthe non-commercial release MORPH-II has become a benchmark data set for age recognition [7]. The MORPH-II data set \ncontains unique images of more than 13,000 individuals. \nAfter the model is built, we test its performance in a 10% holdout set, which will also be used within the experiment \nlater. The performance of the models for age prediction is often evaluated by their mean absolute error (MAE). After \ntraining and optimization procedures, we reach an MAE of ~2.9 on the MORPH-II data set, which is in line with other \nresearchers [1,54]. This means, on average, our model has an error boundary of +/-3 years when predicting the age. \nAs stated above, we generate two fundamentally different types of explanations, a chart showing the probability \ndistribution for each age (“XAI1”, in-model [2]) and an overlay on an image showing particularly relevant parts of the \npicture for the AI’s prediction (“XAI2”, post-model [45]). For the probability distributions, we plot a bar chart that depicts \nthe probabilities—more precisely, the softmax values [56]—for each of the 40 most probable ages. The bars which \ncorrespond to the five most probable ages are highlighted in red. An example of such a bar chart, as presented to the \nparticipants, is depicted in Figure 3. Note that the probabilities are relatively low, which is rooted in the fact that the \nprobabilities for each of the 101 classes add up to 100%. The model often generates somewhat similar probabilities for \nages that are close to each other.","datasource_id":"ds-cmc1ntbw105ug07j49zei4kcb","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null},{"id":"4","source":"test.pdf","page_no":"10","content":"10 \nAs both between-subject and within-subject analyses show significant results, we can support hypothesis 1.1. From an \nanalysis of the boxplot in Figure 5 on p. 11, we see that compliance not only changes but increases with the introduction \nof XAI. Thus, our first finding is: \nFinding 1.1: The introduction of explainability in AI (XAI) increases users’ compliance with the recommendations of \nAI. \nAs our Post Hoc Analysis in Table 2 also reveals, we cannot find significant differences between our treatments \nregarding XAI1 and XAI2. This means we reject hypothesis 1.2. \n \nTable 2: Significance levels of ANOVA and Multiple Comparison of Means with Tukey for Between-subject perspective \n \nCompliance \nANOVA \nAll groups compared \n*** \nMultiple Comparison \nof Means with Tukey \n \nCG ⟷ XAI1 \n*** \nCG ⟷ XAI2 \n*** \nXAI1 ⟷ XAI2 \nn.s. \nNotes: *p < 0.05, **p < 0.01, ***p < 0.001, n.s. = not significant \n \nTable 3: Two-sided t-test comparing compliance with AI before and after treatment \n \nCompliance \nAI1 (Baseline, Stage 1) ⟷ XAI1 (Stage 2) \n*** \nAI2 (Baseline, Stage 1) ⟷ XAI2 (Stage 2) \n* \nNotes: *p < 0.05, **p < 0.01, ***p < 0.001, n.s. = not significant","datasource_id":"ds-cmc1ntbw105ug07j49zei4kcb","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null},{"id":"5","source":"test.pdf","page_no":null,"content":"11 \n \n \nFigure 5: Mean absolute difference (MAD) boxplot of ANOVA for Between-subject perspective of compliance \n5.2 XAI effects on mental model \nWe are not only interested in if and how XAI changes participants’ compliance with the recommendations of AI, but we \nalso investigate potential changes in their MMs. To do so, we first need to set a few statistical prerequisites to ensure the \neligibility of our data. To assess the validity and the reliability of our MM construct, we conduct a confirmatory factor \nanalysis and assess the results with respect to multiple measures. As measures for convergent reliability, we examine \nCronbach’s alpha (CA), average variance extracted (AVE) and composite reliability (CR). \nTable 4: Measurement Information for Latent Factors of Mental Model Construct \n \nAs depicted in Table 4, for all included cases, the constructs of MM GOAL, TASK and PROCESS, the CA, AVE, and CR \nare above the recommended thresholds. A confirmatory factor analysis reveals that factor loadings on all items load highly \n(>0.65) on one factor and with low cross-loadings. These findings demonstrate that our constructs are robust and can be \n \nControl group w/o XAI \nTreatment with XAI1 \nTreatment with XAI2 \nGOAL \nTASK \nPROC \nGOAL \nTASK \nPROC \nGOAL \nTASK \nPROC \n1st order \nReliability \nCA \n0.825 \n0.950 \n0.894 \n0.813 \n0.945 \n0.904 \n0.876 \n0.939 \n0.902 \nCR \n0.832 \n0.951 \n0.894 \n0.816 \n0.945 \n0.905 \n0.879 \n0.939 \n0.902 \nAVE \n0.625 \n0.866 \n0.739 \n0.597 \n0.851 \n0.762 \n0.709 \n0.838 \n0.755 \n2nd order \nReliability \nCR \nMM: \n0.705 \nMM: \n0.757 \nMM: \n0.772 \nNotes: CA = Cronbach’s alpha, CR = composite reliability, AVE = average variance extracted","datasource_id":"ds-cmc1ntbw105ug07j49zei4kcb","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null},{"id":"6","source":"test.pdf","page_no":null,"content":"16 \nstatistical figures and language, it will be useful to conduct future experiments where simple explanations using plain \neveryday language are utilized to test whether they assist in enhancing compliance of users with low AI literacy. \nBeyond the discovery of the above two phenomena, our study further extends the work in IS on MMs. Existing literature \nin IS has mostly focused on how to measure MMs, while the impact of MMs on actual user behavior is scarce so far. With \nour findings, we increase the understanding of how MMs influence human-AI-interaction, more specifically, their impact \non the compliance of users with AI’s recommendations. With this new understanding, we emphasize that users' MM is a \nvariable that researchers and practitioners need to consider when designing and introducing AI. \n7 CONCLUSION \nThe importance of AI-based systems is on the rise. However, more exploration into the relationship between humans and \nAI systems is needed, especially to understand the impact of explanations on users’ compliance with the AI \nrecommendations. \nIn the current study, we elaborate on the relationship between different explainable AI (XAI) methods, users’ AI \nLiteracy, MMSs, and compliance with AI recommendations. We layout a research model and an experimental survey \nsetup. We perform a study with 562 participants who estimate the age of multiple persons—once with the help of an AI \nand once with different treatments of XAI. \nOur overall results show that people’s compliance with the recommendation of AI increases with the introduction of \nXAI. Furthermore, we demonstrate that the introduction of XAI changes users’ MMs of AI. As analyzed with our full \nstructured equation model, the mental model, in turn, significantly influences users’ compliance with the recommendation \nof AI as well. As MMs originate from the background and experience of people, it is not surprising that their AI Literacy, \ni.e., their AI skills and usage, influences their compliance with the recommendations of AI as well. In a subsequent analysis, \nwe even find that by differentiating participants into “low” and “high” AI Literacy groups, we can identify that XAI plays \ndifferent roles for these groups; The type of XAI has no effect on the compliance of participants with low AI Literacy. \nHowever, for participants with high AI Literacy, the type of XAI played a significant role. \nWith these insights, we contribute to the body of knowledge by shedding more light on the relationships between XAI \nand compliance and the related personal characteristics of the users. We show the importance of personalizing XAI as a \nfunction of users’ background and experience, i.e., their AI Literacy. We believe this article should start a debate on the \nnecessity of personalized XAI (PXAI). For instance, in the case of medicine, certain types of XAI—like the visual \nexplanation XAI2 from our treatment—might help doctors better understand the recommendation of an AI-based systems. \nHowever, when presenting explanations to different patients, different XAI techniques might be required, as their MMs \nand expertise are probably at different levels. Therefore, we believe PXAI should be the next frontier in user-centric XAI \nresearch. \nThe generalizability of these results is subject to certain limitations. For instance, other relationships might exist that \nwe did not model in our setup. An additional restrictive factor A limitation of this study is the fact that we only included \none use case with two different types of XAI. Future work needs to implement additional use cases, especially within \nspecialized domains like medicine, and also study the impact of other XAI techniques. Such work would further deepen \nour understanding of the influence of XAI on compliance—and might also help to shed more light on the role of the mental \nmodel. A promising field of research lies ahead.","datasource_id":"ds-cmc1ntbw105ug07j49zei4kcb","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null},{"id":"7","source":"test.pdf","page_no":null,"content":"12 \nfurther used in the upcoming analyses. To examine if and how the MMs of participants change with the introduction of \nXAI with a statistical test, we first need to test for measurement invariance. Measurement invariance is a statistical property \nof measurement that indicates that the same construct is being measured across some specified groups. Precisely, this \nmeans we need to eliminate the possibility that changes in the latent variable between measurement occasions (before and \nafter the treatment) are not attributed to actual change in the latent construct. In the case of an experimental study, this \nmeans we need to eliminate a change in the “psychometric” properties of the measurement instrument, i.e., the construct \nhad a different meaning for the participants at measurement occasions. We test the construct of MM, consisting of the \nsubconstructs GOAL, TASK, and PROCESS for metric, scalar, and strict invariance. To compare the means, we require \nat least scalar invariance [44]. \nIn our case, both metric and scalar invariance are not significant, while strict invariance is significant at the 0.05 level. \nThis means we can compare the latent means of the constructs from before and after the treatment. The results of this \ncomparison are depicted in Table 5. \nThe values show that the MM changes significantly with the introduction of XAI. For XAI1, the constructs TASK and \nPROCESS increase by 0.172 and 0.240, respectively. In the case of XAI2, the PROCESS construct changes significantly; \nmore precisely, it increases by 0.3. We can observe no significant change in the GOAL construct, which is, however, not \nsurprising, as the goal of the decision task didn’t change. In summary, we can support hypothesis 2.1 and conclude: \nFinding 2.1: The introduction of XAI has a positive association with users’ mental models of AI. \nTable 5: Comparison of latent mean differences across measurement occasions with Wald-Test \n5.3 Analysis of mental model and AI Literacy \nWe estimate a full structural equation model (SEM) to better understand the interplay of the different variables considered \nin our study, we estimate a full structural equation model (SEM). Besides the mental model (MM), compliance (COMP), \nand the type of XAI (XAI_T), we also include AI Literacy (AILIT). AI Literacy is modeled as the sum of AI Skills and AI \nUsage. A correlation analysis of the control group reveals that AILIT, MM, and COMP are not considerably correlated \n(<0.3). Table 6 provides the assessment of our model fit. All indices except for Chi-square are within their required \nTreatment \nConstruct \nEstimate \nSE \nz-value \nStd.lv \nStd.all \nXAI1 \nGOAL \n0.029 \n0.061 \n0.467 \n0.027 \n0.027 \nTASK \n0.168** \n0.055 \n3.036 \n0.172 \n0.172 \nPROC \n0.229*** \n0.050 \n4.601 \n0.240 \n0.240 \nXAI2 \nGOAL \n0.063 \n0.072 \n0.873 \n0.061 \n0.061 \nTASK \n0.093 \n0.068 \n1.362 \n0.091 \n0.091 \nPROC \n0.294*** \n0.058 \n5.051 \n0.300 \n0.300 \nNotes: *p < 0.05, **p < 0.01, ***p < 0.001, SE = standard error, Std.lv = standardized estimates (latent), Std.lv = standardized \nestimates (all)","datasource_id":"ds-cmc1ntbw105ug07j49zei4kcb","dataset_id":"dset-cmc1nh2e2lqf507retfodc0dn","file_type":".pdf","external_id":null}]},"finish_reason":null,"index":0}],"created":1750234596,"group_id":"f5e680eb-3762-481b-826f-483a8e74e268","group_name":"Search summary","stage":"Analyze"} id:f5e680eb-3762-481b-826f-483a8e74e268 event:TASK data:{"id":"f5e680eb-3762-481b-826f-483a8e74e268","model":"","choices":[{"delta":{"content":{"name":"Search summary","id":"f5e680eb-3762-481b-826f-483a8e74e268","status":"done","parent_id":null,"stage":"Analyze","properties":{}}},"finish_reason":null,"index":0}],"created":1750234596,"group_id":"f5e680eb-3762-481b-826f-483a8e74e268","group_name":"Search summary","stage":"Analyze"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:TASK data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"content":{"name":"Answer","id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","status":"running","parent_id":null,"stage":"Respond","properties":{}}},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"Okay, let's try a different perspective and provide more details about the dataset, focusing on its potential uses and limitations:\n"},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"**Dataset as a Tool for Understanding Human-AI Interaction:**\n"},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"Instead of just listing the components, let's frame the dataset as a tool for researchers and practitioners interested in building *trustworthy* AI systems. The core value of this dataset lies in its ability to:\n"},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234597,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Quantify the impact of XAI:** The dataset allows researchers to measure how different XAI techniques (the probability distribution chart vs. the image overlay) affect user compliance with AI recommendations. This is crucial for determining which types of explanations are most effective in different contexts.\n"},"finish_reason":null,"index":0}],"created":1750234598,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Uncover the role of AI literacy:** By including measures of AI skills and usage, the dataset enables analysis of how prior knowledge and experience with AI influence a user's response to explanations. This is vital for tailoring XAI to specific user groups.\n"},"finish_reason":null,"index":0}],"created":1750234598,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Model the influence of mental models:** The inclusion of a mental model construct allows researchers to investigate how XAI shapes users' understanding of how the AI works, and how this understanding, in turn, affects their willingness to follow the AI's advice. This provides a deeper understanding of the cognitive processes involved in human-AI collaboration.\n"},"finish_reason":null,"index":0}],"created":1750234598,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Inform the design of personalized XAI (PXAI):** The findings from analyzing this dataset can be used to develop personalized XAI systems that adapt explanations based on a user's AI literacy and mental model. This is a key step towards building AI that is both effective and understandable.\n"},"finish_reason":null,"index":0}],"created":1750234599,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234599,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"**Limitations and Considerations:**\n"},"finish_reason":null,"index":0}],"created":1750234599,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234599,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"It's also important to acknowledge the limitations of the dataset:\n"},"finish_reason":null,"index":0}],"created":1750234599,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234599,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} :keep-alive id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Single Use Case:** The study focuses on a single task (age estimation) and a limited set of XAI techniques. The results may not generalize to other domains or other types of explanations. As the original paper mentions, future work should include additional use cases, especially within specialized domains like medicine.\n"},"finish_reason":null,"index":0}],"created":1750234600,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Image Quality and Demographics:** While the MORPH dataset addresses some issues with the IMDB-WIKI dataset, there might still be biases related to the demographics represented in the images. The dataset should be carefully examined for potential biases before being used to train or evaluate AI systems.\n"},"finish_reason":null,"index":0}],"created":1750234600,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Online Experiment Setting:** The online experiment setting may introduce biases related to participant attention and motivation. The use of attention checks helps to mitigate this, but it's still a factor to consider.\n"},"finish_reason":null,"index":0}],"created":1750234600,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Self-Reported Measures:** The measures of AI literacy and mental models are based on self-reported data, which may be subject to biases.\n"},"finish_reason":null,"index":0}],"created":1750234601,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"* **Specific XAI Implementations:** The specific implementations of XAI1 (probability distribution) and XAI2 (LIME overlay) might influence the results. Different implementations of these techniques could lead to different outcomes.\n"},"finish_reason":null,"index":0}],"created":1750234601,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"\n"},"finish_reason":null,"index":0}],"created":1750234601,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:MESSAGE data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"role":null,"content":"**In summary:** This dataset is a valuable resource for studying the impact of XAI on user compliance and understanding. However, it's crucial to be aware of its limitations and to interpret the results in the context of the specific task, XAI techniques, and participant population used in the study. The dataset provides a foundation for further research into personalized and trustworthy AI.\n"},"finish_reason":null,"index":0}],"created":1750234601,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} id:abab8d59-57d6-4e4e-8454-a03a264fb04b event:TASK data:{"id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","model":"","choices":[{"delta":{"content":{"name":"Answer","id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","status":"done","parent_id":null,"stage":"Respond","properties":{}}},"finish_reason":null,"index":0}],"created":1750234601,"group_id":"abab8d59-57d6-4e4e-8454-a03a264fb04b","group_name":"Answer","stage":"Respond"} event:END_MARK data:[DONE] ```
**Example request**: ```curl cURL theme={null} curl --request POST \ --url https://ai.data.cloud/api/v2/team/jobs \ --header 'Content-Type: application/json' \ --header 'x-pd-api-key: $PD_API_KEY' \ --data '{ "session_id": "bc9a8127-4214-42b2-bbbe-a022f23d9795", "user_id": "$UID", "stream": false, "question": "introducing the dataset", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "datasource_ids": [ "ds-cmc1ntbw105ug07j49zei4kcb" ], "output_language": "EN", "job_mode": "AUTO" }' ``` ```python Python theme={null} import requests url = "https://ai.data.cloud/api/v2/team/jobs" payload = { "session_id": "bc9a8127-4214-42b2-bbbe-a022f23d9795", "user_id": "$UID", "stream": False, "question": "introducing the dataset", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "datasource_ids": ["ds-cmc1ntbw105ug07j49zei4kcb"], "output_language": "EN", "job_mode": "AUTO" } headers = { "x-pd-api-key": "$PD_API_KEY", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) ```
```json theme={null} { "code": 0, "data": { "blocks": [ { "type": "MESSAGE", "content": "Okay, let's introduce the dataset from the perspective of **Explainable AI (XAI) methodology and the evaluation of explanation quality:**\n\n**Dataset as a Benchmark for Evaluating XAI Techniques and Explanation Quality:**\n\nThis dataset provides a valuable resource for researchers working on Explainable AI (XAI) methodologies. It allows for the evaluation and comparison of different XAI techniques based on their impact on user behavior and understanding. Instead of just focusing on user compliance, we can use this dataset to assess the *quality* of the explanations themselves.\n\n* **Comparing XAI Techniques:** The dataset directly compares two different XAI techniques (probability distribution and image overlay) in the context of age estimation. This allows researchers to assess the strengths and weaknesses of each technique in terms of:\n * **Comprehensibility:** How easily can users understand the explanation?\n * **Faithfulness:** How accurately does the explanation reflect the AI's decision-making process?\n * **Sufficiency:** Does the explanation provide enough information for users to make informed decisions?\n * **Necessity:** Does the explanation contain only the information that is necessary for users to understand the AI's decision?\n* **Developing New XAI Metrics:** The dataset can be used to develop and validate new metrics for evaluating the quality of XAI explanations. These metrics could be based on:\n * **User Understanding:** Measuring how well users can answer questions about the AI's decision-making process after seeing the explanation.\n * **User Trust:** Measuring how much users trust the AI's recommendations after seeing the explanation.\n * **Decision Quality:** Measuring how well users perform on the age estimation task after seeing the explanation.\n* **Investigating the Relationship Between Explanation Quality and User Behavior:** The dataset allows researchers to explore the relationship between different aspects of explanation quality (e.g., comprehensibility, faithfulness) and user behavior (e.g., compliance, trust, decision quality). Which aspects of explanation quality are most important for influencing user behavior?\n* **Benchmarking XAI Algorithms:** The dataset can serve as a benchmark for evaluating the performance of different XAI algorithms. Researchers can use the dataset to compare the explanations generated by different algorithms in terms of their quality and impact on user behavior.\n* **Exploring the Impact of Explanation Fidelity:** How accurately does the explanation reflect the true reasoning of the AI model? While the dataset doesn't directly measure fidelity, it provides a context to infer it. For example, if users with high AI literacy *decrease* compliance despite seeing explanations, it might suggest the explanations are not faithful to the underlying model.\n* **Analyzing Explanation Effectiveness Across Different User Groups:** Does the effectiveness of different XAI techniques vary depending on the user's AI literacy, cognitive abilities, or other characteristics? The dataset allows researchers to investigate these questions.\n\n**Key XAI Methodology Considerations:**\n\n* **Formal Definitions of Explanation Quality:** Researchers should strive to develop formal definitions of explanation quality that are grounded in theory and empirically validated.\n* **Objective Evaluation Metrics:** The evaluation of XAI techniques should rely on objective metrics whenever possible, rather than solely on subjective user ratings.\n* **Human-Centered Evaluation:** XAI techniques should be evaluated in the context of real-world tasks and with real users.\n* **Iterative Design and Evaluation:** The design and evaluation of XAI techniques should be an iterative process, with feedback from users informing the development of new and improved explanations.\n\n**In summary:** This dataset provides a valuable resource for advancing the field of Explainable AI by enabling the rigorous evaluation and comparison of different XAI techniques. By focusing on explanation quality, developing new evaluation metrics, and understanding the relationship between explanation quality and user behavior, researchers can contribute to the development of more effective and trustworthy AI systems. This perspective emphasizes the importance of not just providing explanations, but of providing *good* explanations that are truly helpful to users.\n", "stage": "Respond", "group_id": "5538c8d3-5acd-4831-b40d-37fbdbeb5071", "group_name": "Answer" }, { "type": "SOURCES", "content": [ { "id": "1", "source": "test.pdf", "content": "summary: test.pdf An experiment with 562 participants investigated the impact of Explainable AI (XAI) and AI literacy on user compliance. Results revealed that XAI boosts compliance, influenced by AI literacy, with the relationship mediated by users' mental model of AI. This study highlights the importance of XAI in AI-based system design. It explores the connection between AI literacy, mental models, XAI techniques, and user compliance with AI recommendations. The research also examines the effect of presenting different XAI types on user compliance. An AI artifact was developed to predict age from photographs, offering personalized explanations to enhance decision-making and compliance with AI recommendations. The study delves into AI interpretability, AI literacy, explainable AI models, and their influence on user behavior. It discusses advancements in AI, machine learning, and user interaction, addressing areas like facial recognition, digital resilience, and algorithmic fairness.", "page_no": "", "datasource_id": "ds-cmc1ntbw105ug07j49zei4kcb", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" }, { "id": "2", "source": "test.pdf", "content": "summary: test.pdf An experiment with 562 participants investigated the impact of Explainable AI (XAI) and AI literacy on user compliance. Results revealed that XAI boosts compliance, influenced by AI literacy, with the relationship mediated by users' mental model of AI. This study highlights the importance of designing AI systems with XAI for better user engagement.", "page_no": "", "datasource_id": "ds-cmc1nrv4a05ue07j4vscij2z7", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" }, { "id": "3", "source": "test.pdf", "content": "7 The decision for a data set for building an AI for age estimation is tightly bound to the current research basis on ML models for age estimation. Age estimation has been of particular interest in the ML community, and many researchers have tackled the task of predicting the age of a person on an image [11,47]. The largest and most popular data set is the IMDB- WIKI data set [47], which we utilize for training our AI. For our implementation, we take advantage of the source code published by Serengil [51], with minor adjustments in Python, using the popular keras package. The model itself is based on a CNN, which uses the VGG-16 architecture and is pre-trained on the FaceNet database [50]. The network architecture is then adjusted to the age estimation task and our specific data set. While the IMDB-WIKI data set is widely used as a training basis for age estimation models and the use of existing, published models makes them convenient to use, there are multiple reasons for which the pictures in this data set cannot be used for display (≠ model training) in our study; the quality of the images varies vastly, the ages of the persons are not validated, and the data set contains many pictures of celebrities. Especially the latter could falsify the participants' performance as they might have existing knowledge of the age of a person. Another factor that might have an unintended effect on the study is the fact that the images are taken “in the wild”, meaning that there is no standard way of how the people are shown in the image. The people are pictured in various ways, with different poses, facial expressions like smiles or laughter, and clothing like sunglasses, headgear, or jewelry. To address these shortcomings, we use the MORPH data set Feld for model adoption and presentation to the study participants[46]. It has been specifically developed for research purposes and contains the actual age of the people depicted in the pictures. While there are multiple versions of MORPH, the non-commercial release MORPH-II has become a benchmark data set for age recognition [7]. The MORPH-II data set contains unique images of more than 13,000 individuals. After the model is built, we test its performance in a 10% holdout set, which will also be used within the experiment later. The performance of the models for age prediction is often evaluated by their mean absolute error (MAE). After training and optimization procedures, we reach an MAE of ~2.9 on the MORPH-II data set, which is in line with other researchers [1,54]. This means, on average, our model has an error boundary of +/-3 years when predicting the age. As stated above, we generate two fundamentally different types of explanations, a chart showing the probability distribution for each age (“XAI1”, in-model [2]) and an overlay on an image showing particularly relevant parts of the picture for the AI’s prediction (“XAI2”, post-model [45]). For the probability distributions, we plot a bar chart that depicts the probabilities—more precisely, the softmax values [56]—for each of the 40 most probable ages. The bars which correspond to the five most probable ages are highlighted in red. An example of such a bar chart, as presented to the participants, is depicted in Figure 3. Note that the probabilities are relatively low, which is rooted in the fact that the probabilities for each of the 101 classes add up to 100%. The model often generates somewhat similar probabilities for ages that are close to each other.", "page_no": "7", "datasource_id": "ds-cmc1ntbw105ug07j49zei4kcb", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" }, { "id": "4", "source": "test.pdf", "content": "10 As both between-subject and within-subject analyses show significant results, we can support hypothesis 1.1. From an analysis of the boxplot in Figure 5 on p. 11, we see that compliance not only changes but increases with the introduction of XAI. Thus, our first finding is: Finding 1.1: The introduction of explainability in AI (XAI) increases users’ compliance with the recommendations of AI. As our Post Hoc Analysis in Table 2 also reveals, we cannot find significant differences between our treatments regarding XAI1 and XAI2. This means we reject hypothesis 1.2. Table 2: Significance levels of ANOVA and Multiple Comparison of Means with Tukey for Between-subject perspective Compliance ANOVA All groups compared *** Multiple Comparison of Means with Tukey CG ⟷ XAI1 *** CG ⟷ XAI2 *** XAI1 ⟷ XAI2 n.s. Notes: *p < 0.05, **p < 0.01, ***p < 0.001, n.s. = not significant Table 3: Two-sided t-test comparing compliance with AI before and after treatment Compliance AI1 (Baseline, Stage 1) ⟷ XAI1 (Stage 2) *** AI2 (Baseline, Stage 1) ⟷ XAI2 (Stage 2) * Notes: *p < 0.05, **p < 0.01, ***p < 0.001, n.s. = not significant", "page_no": "10", "datasource_id": "ds-cmc1ntbw105ug07j49zei4kcb", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" }, { "id": "5", "source": "test.pdf", "content": "15 detail, we find two interesting phenomena. First, we find that AI literacy impacts compliance positively when it is mediated through MM; however, negatively when the impact is measured directly from AI literacy on compliance. Second, we find that the compliance with AI recommendations of users with low AI literacy is not impacted by the explanations provided to them. Figure 7: Summary of Findings The first phenomenon uncovered highlights a paradox where AI Literacy reduces compliance on the one hand but improves MM (which then improves compliance) on the other. This phenomenon is also referred to as inconsistent mediation [37]. We believe that an increase in AI literacy impacts MMs because the increase in skills and experience with AI allows for different, potentially more precise, mental representations of AI (as compared to without that skill and experience). Having said this, in accordance with the algorithm aversion theory, the individuals with a higher level of AI literacy also understand the imperfections in the AI models. This knowledge of imperfections in AI models may decrease their trust in the AI recommendations. With a lower level of trust in AI models, individuals with high AI literacy may tend to trust their own judgment than complying with AI’s recommendations. The fact that the SEM model shows an increase in compliance with shifts in MMs but a decrease in compliance directly highlights a tension in the minds of individuals with high AI literacy. They must constantly balance between trusting the precision brought through an AI model or mistrusting the imperfections inherent in the AI model. The second phenomenon uncovered applies to users with low AI literacy, and the lack of impact of explanations (XA1 and XAI2) on their compliance with AI’s recommendations. We made this discovery while conducting subsample analyses of the AI Literacy construct. We found that while the type of XAI (i.e., XAI1 or XAI2) has a significant effect on compliance for participants with high AI Literacy, this does not hold true for participants with low AI Literacy. This means that in terms of compliance, it does not make a difference for participants with low AI Literacy as to which type of XAI is presented to them. It appears that participants with low AI literacy do not know what to do with the explanations provided to them. Instead, their compliance with AI recommendations is only impacted by their MMs. This finding is a proof point for our call to design more user-centric, more personalized explanations in AI (PXAI). With more personalized explanations, the AI practitioner community can potentially develop explanations that serve users who have no education in AI or statistics to understand even the most basic statistical charts or figures. Since both types of explanations tested within our study used AI Explainability (experimental Treatment: None/XAI Type1/ XAI Type 2) Compliance with AI (+) (+) (+) (-) (+) Mental Model AI Literacy", "page_no": "15", "datasource_id": "ds-cmc1ntbw105ug07j49zei4kcb", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" }, { "id": "6", "source": "test.pdf", "content": "11 Figure 5: Mean absolute difference (MAD) boxplot of ANOVA for Between-subject perspective of compliance 5.2 XAI effects on mental model We are not only interested in if and how XAI changes participants’ compliance with the recommendations of AI, but we also investigate potential changes in their MMs. To do so, we first need to set a few statistical prerequisites to ensure the eligibility of our data. To assess the validity and the reliability of our MM construct, we conduct a confirmatory factor analysis and assess the results with respect to multiple measures. As measures for convergent reliability, we examine Cronbach’s alpha (CA), average variance extracted (AVE) and composite reliability (CR). Table 4: Measurement Information for Latent Factors of Mental Model Construct As depicted in Table 4, for all included cases, the constructs of MM GOAL, TASK and PROCESS, the CA, AVE, and CR are above the recommended thresholds. A confirmatory factor analysis reveals that factor loadings on all items load highly (>0.65) on one factor and with low cross-loadings. These findings demonstrate that our constructs are robust and can be Control group w/o XAI Treatment with XAI1 Treatment with XAI2 GOAL TASK PROC GOAL TASK PROC GOAL TASK PROC 1st order Reliability CA 0.825 0.950 0.894 0.813 0.945 0.904 0.876 0.939 0.902 CR 0.832 0.951 0.894 0.816 0.945 0.905 0.879 0.939 0.902 AVE 0.625 0.866 0.739 0.597 0.851 0.762 0.709 0.838 0.755 2nd order Reliability CR MM: 0.705 MM: 0.757 MM: 0.772 Notes: CA = Cronbach’s alpha, CR = composite reliability, AVE = average variance extracted", "page_no": "", "datasource_id": "ds-cmc1ntbw105ug07j49zei4kcb", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" }, { "id": "7", "source": "test.pdf", "content": "12 further used in the upcoming analyses. To examine if and how the MMs of participants change with the introduction of XAI with a statistical test, we first need to test for measurement invariance. Measurement invariance is a statistical property of measurement that indicates that the same construct is being measured across some specified groups. Precisely, this means we need to eliminate the possibility that changes in the latent variable between measurement occasions (before and after the treatment) are not attributed to actual change in the latent construct. In the case of an experimental study, this means we need to eliminate a change in the “psychometric” properties of the measurement instrument, i.e., the construct had a different meaning for the participants at measurement occasions. We test the construct of MM, consisting of the subconstructs GOAL, TASK, and PROCESS for metric, scalar, and strict invariance. To compare the means, we require at least scalar invariance [44]. In our case, both metric and scalar invariance are not significant, while strict invariance is significant at the 0.05 level. This means we can compare the latent means of the constructs from before and after the treatment. The results of this comparison are depicted in Table 5. The values show that the MM changes significantly with the introduction of XAI. For XAI1, the constructs TASK and PROCESS increase by 0.172 and 0.240, respectively. In the case of XAI2, the PROCESS construct changes significantly; more precisely, it increases by 0.3. We can observe no significant change in the GOAL construct, which is, however, not surprising, as the goal of the decision task didn’t change. In summary, we can support hypothesis 2.1 and conclude: Finding 2.1: The introduction of XAI has a positive association with users’ mental models of AI. Table 5: Comparison of latent mean differences across measurement occasions with Wald-Test 5.3 Analysis of mental model and AI Literacy We estimate a full structural equation model (SEM) to better understand the interplay of the different variables considered in our study, we estimate a full structural equation model (SEM). Besides the mental model (MM), compliance (COMP), and the type of XAI (XAI_T), we also include AI Literacy (AILIT). AI Literacy is modeled as the sum of AI Skills and AI Usage. A correlation analysis of the control group reveals that AILIT, MM, and COMP are not considerably correlated (<0.3). Table 6 provides the assessment of our model fit. All indices except for Chi-square are within their required Treatment Construct Estimate SE z-value Std.lv Std.all XAI1 GOAL 0.029 0.061 0.467 0.027 0.027 TASK 0.168** 0.055 3.036 0.172 0.172 PROC 0.229*** 0.050 4.601 0.240 0.240 XAI2 GOAL 0.063 0.072 0.873 0.061 0.061 TASK 0.093 0.068 1.362 0.091 0.091 PROC 0.294*** 0.058 5.051 0.300 0.300 Notes: *p < 0.05, **p < 0.01, ***p < 0.001, SE = standard error, Std.lv = standardized estimates (latent), Std.lv = standardized estimates (all)", "page_no": "", "datasource_id": "ds-cmc1ntbw105ug07j49zei4kcb", "dataset_id": "dset-cmc1nh2e2lqf507retfodc0dn", "file_type": ".pdf" } ], "stage": "Analyze", "group_id": "49f27d13-6b8b-4449-951e-eb0861932615", "group_name": "Search summary" } ], "job_id": "job-cmc1qop11mc5007replx25nk3" } } ```
*** ## Need more help? Get answers from our members Tell us more and we'll help you out # How to Upload Local Files Source: https://docs.powerdrill.ai/developer-guides/upload-file How to quickly upload local files to create data sources The [`POST api/v1/datasets/{datasetId}/datasources`](/api-reference/create-data-source) and [`POST /v1/datasources`](/api-reference/create-data-source-without-dataset) endpoints allow you to upload your data in two methods: * From a public URL: Use the `url` parameter. * From your local storage: Use the `fileKey` parameter. This guide focuses on the latter—uploading a local file to create a data source. *** ## Authentication All requests to Powerdrill must include an `x-pd-api-key` header with your API key. To get your API key, see [Quick Start](https://docs.powerdrill.ai/developer-guides/quick-start#step-1-get-your-api-key). *** ## Step 1. Upload your local file and obtain the `fileKey` Use the [POST /v1/file/upload\_datasource](/api-reference/upload-file) endpoint to upload your local file. After the upload, the response will include a `fileKey`. Save this `fileKey` to create a data source using the [Create data source](/api-reference/create-data-source) endpoint. Supported file formats include: **.csv**, **.tsv**, **.md**, **.mdx**, **.json**, **.txt**, **.pdf**, **.pptx**, **.ppt**, **.doc**, **.docx**, **.xls**, and **.xlsx**. Here's an example in cURL: ```curl Example request: theme={null} curl --location 'http://ai.data.cloud/api/v1/file/upload_datasource' \ --header 'Content-Type: multipart/form-data' \ --header 'x-pd-api-key: ' \ --header 'x-pd-api-agent-id: GENERAL' \ --form 'file=@""' ``` When making a request: * Replace `` with your actual API key. * Replace `` with the actual full path to your file, for example, `/Users/test/workspace/sales_2024.csv`. Here's an example response: ```json Example response: theme={null} { "code": 0, "data": { "fileKey": "/tmp/sdgsagdsgsadgasdg" } } ``` Extract the `fileKey` value from the response for later use. In this example, it is `/tmp/sdgsagdsgsadgasdg`. *** ## Step 2. Create a data source Now, create a data source to embed and synchronize it with your AI Workspace. You can do this by sending a request to either the [`POST api/v1/datasets/{datasetId}/datasources`](/api-reference/create-data-source) endpoint or the [`POST /v1/datasources`](/api-reference/create-data-source-without-dataset) endpoint. The example below demonstrates the use of the [`POST /v1/datasources`](/api-reference/create-data-source-without-dataset) endpoint. ```curl Example request: theme={null} curl --location 'https://ai.data.cloud/api/v1/datasets/cm3my37en3q36017q7x3hyyf4/datasources' \ --header 'x-pd-api-key: $PD_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "name": "test.csv", "fileName": "test.csv", "type": "FILE", "fileKey": "/tmp/sdgsagdsgsadgasdg" }' ``` When making a request: * Replace `` with your actual API key. * Set `fileKey` to your actual one. ```json Example response: theme={null} { "code": 0, "data": { "id": "datasource-cadsgfsdagasgadsg", "datasetId": "dataset-dagasdgasgasg", "name": "test.csv", "fileName": "test.csv", "type": "FILE", "status": "pending" } } ``` Obtain the `id` (data source ID) and `datasetId` (dataset ID) for later use. In this example, the data source ID is `datasource-cadsgfsdagasgadsg` and the dataset ID is `dataset-dagasdgasgasg`. *** ## Step 3. Check data source status The data source must be in the **synched** state to be ready for use. To check its status, use the [`GET v1/datasets/{datasetId}/datasources/{datasourceId}`](/api-reference/get-data-source) endpoint. Here's an example: ```curl Example request theme={null} curl --request GET \ --url https://ai.data.cloud/api/v1/datasets/dataset-dagasdgasgasg/datasources/datasource-cadsgfsdagasgadsg \ --header 'x-pd-api-key: ' ``` Check the status in the response: ```json Example response: theme={null} { "code": 0, "data": { "id": "datasource-cadsgfsdagasgadsg", "datasetId": "dataset-dagasdgasgasg", "name": "test.csv", "fileName": "test.csv", "type": "FILE", "status": "synched" } } ``` In this example, the data source is in the `synched` state and is ready for use in [data analysis jobs](/api-reference/create-job). Other possible statuses include: * **`pending`**: Waiting to be processed. * **`running`**: Currently being processed. * **`error`**: Processing failed. If the status is `pending` or `running`, wait for some time and check again until it changes to `synched`. If the status is `error`, you'll need to re-upload the file by starting from [Step 1](#step-1-upload-your-local-file-and-obtain-fileKey). *** ## Need more help? Get answers from our members Tell us more and we'll help you out # How Can I Delete My Account After I Have Created a Team on Powerdrill Enterprise? Source: https://docs.powerdrill.ai/enterprise/delete-account The admin account is closely tied to your organization. Do not delete it unless necessary Deleting the admin account is a critical action, as it is closely tied to your organization. To proceed, you must first delete your entire organization. We strongly advise against this operation unless it is absolutely necessary. Please carefully consider the impacts before taking this operation. If you still wish to delete your admin account, follow the steps below. *** ## Step 1. Delete your organization Deleting your organization is a serious action and cannot be undone. We strongly recommend carefully considering whether it's absolutely necessary before proceeding. Please make sure this is the right decision for your team and data. Once you delete your organization, the following will be lost: * All users will be removed from the team. * All subscription records will be deleted. * All projects and datasets within those projects will be removed. * Credit card information will be deleted. * Usage and billing data will be erased. If you still wish to delete your organization, please follow these steps in the Admin console: Don't know how to enter the Admin console? Refer to this [FAQ](/enterprise/enter-admin-console.mdx). 1. Cancel all subscriptions to job plans (if any): Go to the **Subscriptions & plans** page, choose **Manage** > **Unsubscribe** in the **Actions** column for each subscription, and confirm the operation. 2. Cancel your subscription to the AI Workspace capacity plan (if any): Go to the **Usage & billing** page, click **Unsubscribe** in the **Workspace capacity** section. 3. Delete your organization: Go to the **Settings** page, click **Delete organization** in the **DANGER ZONE** section, and confirm the action. *** ## Step 2. Unsubcribe from your Powerdrill Personal Edition Plan After deleting your organization, check if you are subscribed to any pricing plan under the Powerdrill Personal Edition. If so, make sure to unsubscribe before proceeding. If not, skip this step. 1. Click your profile icon on the upper-right corner of the page and click **Switch Workspace**. 2. On the page that appears, click the **My personal space** card. 3. In your personal space that appears, click your profile icon on the lower-right corner. 4. Click **Upgrade & renew**, choose **...** > **Unsubscribe** next to **My Invoices** button, and then confirm your unsubscription as prompted. *** ## Step 3. Delete your account Once there is no organization or personal edition plan linked to your account, you may proceed to delete your organization. We want to remind you once again that this action is not recommended. Please carefully consider the impacts before proceeding. Once you delete your account, all data—including your datasets and job execution history—will be permanently erased and cannot be recovered. 1. Click **My account**. 2. In the **DANGER ZONE** section, click **Delete account**, and confirm the operation. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # How Can I Enter the Admin Console of My Team? Source: https://docs.powerdrill.ai/enterprise/enter-admin-console The Admin console is where you manage the projects, users, subscriptions, and settings in your team To manage your team's projects, users, subscriptions, and settings using the admin account, you must first enter the Admin console: 1. Sign in to [Powerdrill](https://powerdrill.ai). 2. Click your profile icon in the lower-left corner. 3. In the menu, go to the **Workspace** section and click **Switch Workspace**. 4. Click the **Admin console** to enter the Admin console. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Introduction Source: https://docs.powerdrill.ai/enterprise/introduction A brief introduction about Powerdrill Enterprise and how to choose the right solution ## Introducing Powerdrill Enterprise **Powerdrill Enterprise** is our new solution designed to enhance data-driven strategies with enterprise-grade security, privacy, and advanced AI analytics. It combines powerful data analysis, flexible deployment options, and extensive customization, enabling businesses to achieve unmatched efficiency and insights. Choose from two options: **SaaS Team** for instant access via APIs or **Cloud Deployment** for full control within your secure cloud environment. *** ## SaaS Team The SaaS Team solution offers immediate activation, enabling organizations to quickly create teams and access a wide range of APIs to unlock key features. Designed for scalability and flexibility, it allows teams to build robust knowledge bases and datasets, supporting advanced AI data analysis and data-driven decision-making. **It's ideal for organizations seeking efficient, on-demand access to AI resources with minimal setup, providing a solid foundation for rapid innovation and streamlined AI project collaboration.** Your team serves as your organization's virtual representation in Powerdrill Enterprise. After creating your team, Powerdrill Enterprise automatically generates a default project for adding data, managing users, and running jobs. You can create as many projects as needed. While all projects share a common pool of Workspace capacity, each project operates in its own isolated environment. This ensures users and datasets in one project are inaccessible to others, providing granular control over data security and clear boundaries between projects. *** ## Dedicated Cloud Designed for businesses with strict data security and privacy needs, **this cloud deployment solution integrates seamlessly with your preferred cloud platform, providing AI-powered data analysis in a secure, isolated environment.** With the flexibility to deploy across major cloud providers, it ensures sensitive data remains protected while utilizing advanced AI insights. Enjoy the security of a private cloud infrastructure combined with the scalability to support advanced analytics, making it ideal for enterprises focused on data integrity, performance optimization, and control. *** ## Why Powerdrill Enterprise? ### Easy to integrate * **SaaS Team**: Quickly set up and organize teams while seamlessly developing with Powerdrill's robust API suite, making it easy to integrate AI capabilities into your workflow. * **Dedicated Cloud**: Start effortlessly with Docker for streamlined deployment and leverage scalable cloud resources to grow as your needs expand, ensuring optimal performance and flexibility. * **LLM Compatibility**: Powerdrill Enterprise is compatible with a wide range of proprietary and open-source large language models (LLMs), allowing you to choose the model that best suits your application, from popular open-source options to specialized proprietary LLMs. ### AI-powered serious data work, unlocking 100x efficiency AI supercharges data processing and analysis, achieving speeds up to 100 times faster than traditional methods. This transformative boost enables businesses to rapidly extract insights, streamline operations, and drive data-informed decisions—enhancing efficiency and giving you a strong competitive edge. ### Compliance, governance, and observability In AI data analytics, compliance safeguards adherence to legal and regulatory standards, governance defines robust policies and controls for responsible data management, and observability offers deep insights into system performance and behavior. Together, these pillars create a framework for secure, efficient, and accountable AI use, fostering trust and reliability in data-driven solutions. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Pricing Source: https://docs.powerdrill.ai/enterprise/pricing Pricing plans for Powerdrill Enterprise Powerdrill Enterprise provides flexible pricing options tailored to your solution choice. Before subscribing, consider these two core billing factors, applicable to both **SaaS Team** and **Dedicated Cloud**: * **Jobs**: Each job is completed when a message is sent and a response is receilved, whether through a data agent, API call, or within a private Powerdrill deployment. To get a detailed understanding about a **job**, refer to [What Is Job?](/enterprise/what-is-job.mdx). * **Workspace capacity**: refers to the allocated size of your hybrid store, which includes offline indexing, vector storage and retrieval, and enterprise-grade data security. *** ## Pricing for SaaS Team Edition ### Pricing plans for jobs Choose from the following pricing plans to get job credits for you data agents. | Plan | Quota | Monthly Subscription Price | One-Time Purchase Price (One Month) | | :----- | :------------------- | :------------------------- | :---------------------------------- | | Tier 1 | 200 jobs / agent | \$8.40 | \$9.20 | | Tier 2 | 600 jobs / agent | \$25.20 | \$27.70 | | Tier 3 | 2000 jobs / agent | \$84.00 | \$92.40 | | Tier 4 | 6,000 jobs / agent | \$176.00 | \$194.00 | | Tier 5 | 16,000 jobs / agent | \$470.00 | \$517.00 | | Tier 6 | 50,000 jobs / agent | \$1,470.00 | \$1,617.00 | | Tier 7 | 150,000 jobs / agent | \$4,410.00 | \$4,851.00 | | Tier 8 | 500,000 jobs / agent | \$14,700.00 | \$16,170.00 | In Powerdrill Enterprise, an organization can have multiple users, but each user must be associated with a subscription to run jobs. A subscription can only be linked to one user. **If you want multiple users to execute jobs within the organization, you will need to create a subscription for each user and associate them individually.** Users who are not linked to a subscription will not be able to run jobs within the organization. Each plan provides a set job quota, allowing you to select the option that best fits your operational needs and budget. Powerdrill Enterprise provides 100 free jobs for each new organization. This quota is valid for one month and will expire if not used, being cleared one month after the organization's creation. ### Pricing plans for AI Workspace capacity Choose from the following plans based on the estimated storage size required for your data assets, including files of all types and plain text. | Plan | Quota | Monthly Subscription Price | | :----- | :------ | :------------------------- | | Tier 1 | 500 MB | \$2.00 | | Tier 2 | 1.5 GB | \$5.00 | | Tier 3 | 15 GB | \$45.00 | | Tier 4 | 100 GB | \$180.00 | | Tier 5 | 1000 GB | \$1,080.00 | Each organization gets 100 MB workspace capacity for free after creating a team on Powerdrill Enterprise. Each tier provides a different storage capacity to accommodate your AI Workspace needs, from basic usage to more extensive data storage. The capacity is shared by all projects in your team, but datasets created in one project is isolated from the others. *** ## Pricing for Dedicated Cloud Edition [Fill in the form](https://docs.google.com/forms/d/e/1FAIpQLSfqLaiYDciUkU9WYG8OQyrJTFuN6Xuw1lbbMlRiVbfXOQ1Zdg/viewform), and our sales team will get in touch with you promptly. *** ## FAQ ### Will I get 600 MB or 500 MB of AI Workspace capacity after upgrading to the Tier 1 AI Workspace capacity plan? You will receive 500 MB of Workspace capacity. The Workspace capacity of each tier is included in the total capacity of the next tier, so you won't get additional capacity beyond what Tier 1 offers. ### Will the free job quota expire? Yes, the free job quota will expire one month after your organization is created. ### Will the free AI Workspace capacity expire? No, the free AI Workspace capacity will not expire. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Projects Source: https://docs.powerdrill.ai/enterprise/projects This guide introduces how to create and manage projects in your organization ## What is a project? Projects are focused, purpose-driven groups within your organization. You can create multiple projects based on your specific needs. While all projects share the same pool of Workspace capacity, each project functions as a data-isolated environment. This means datasets created in one project are inaccessible to other projects, enabling you to maintain granular control over data security and ensure clear boundaries between projects. *** ## Create a project 1. Sign in to Powerdrill. 2. Click your profile icon in the lower-left corner. 3. In the menu, go to the **Workspace** section and click **Switch Workspace**. A list of all the workspaces you're in will appear. 4. Click the **Admin console** next to the team that you own. 5. On the top navigation bar, select **Projects**. On the **Projects** page, click **Create a project**. 6. Set the project name and click **Create**. Now you can add users or manage API keys to control access to your project's resources. *** ## Manage project members You can manage the members in each project on the Admin console. ### Add a user to a project You can add users to a project so that they can collaborate in the project. 1. In the Admin console, click **Projects** in the top navigation bar. 2. Select the target project from the project list. 3. On the **Users** tab of the project details page, click **Add user**. 4. In the dialog box that appears, select the users you want to add and click **Add**. ### Remove users from your project You can remove unnecessary members from your project to enhance data security. Removed users will no longer have access to the project's datasets for running data analysis jobs. 1. In the Admin console, click **Projects** in the top navigation bar. 2. Select the target project from the project list. 3. On the **Users** tab of the project details page, locate the user you want to remove and click the Delete icon in the **Actions** column. To remove multiple users at a batch, select the checkboxes next to the IDs of the users you want to remove. 4. In the confirmation dialog that appears, click **Yes, remove**. *** ## Manage project API keys **Create a project API key**: 1. In the Admin Console, click **Projects** in the top navigation bar. 2. Select the target project from the project list. 3. On the project details page, click the **Project API keys** tab. 4. Click **+ API access key**. 5. In the dialog box that appears, enter a name for the API key and click **Create**. 6. Copy the generated secret key, save it securely, and click **I have saved my secret key**. Please save the secret key properly, as it is displayed only once. If you lose it, you can only create a new one. **Delete a project API key**: 1. In the Admin Console, click **Projects** in the top navigation bar. 2. Select the target project from the project list. 3. On the project details page, click the **Project API keys** tab. 4. In the list of API keys, locate the unnecessary project API key, click the Delete icon in the **Actions** column. 5. In the dialog box that appears, click **Yes, revoke** to confirm the deletion. *** ## FAQ ### Can I delete unnecessary projects? No. Currently you cannot delete projects from your team. ### What's the difference between project API keys and team API keys? They serve for different purposes. * **Project API keys**: Obtained from the **Projects** page, they are used for authenticating operations to resources within specific projects. * **Team API keys**: Obtained from the **Settings** page, they are used for authenticating actions related to creating or modifying projects within your team. ### How do I manage datasets and data sources in a project? 1. Click your profile icon, and click **Switch Workspace**. 2. Click the project card to enter the project. Then, you will see the homepage of the project. 3. In the left sidebar, select **Datasets**. From there, you can manage your datasets and data sources in each dataset as needed. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Quick Start Source: https://docs.powerdrill.ai/enterprise/quick-start Get started with Powerdrill Enterprise This guide will walk you through the essential steps to set up your team, associate your user with a subscription, and generate an API key for your project. Whether you're an admin or a developer, you'll be up and running in just a few minutes. ## Step 1. Create a team 1. Sign in to Powerdrill. 2. Click your profile icon in the lower-left corner. 3. In the menu, go to the **Workspace** section and click **Create team**. 4. Enter your organization name, agree to the *Terms of Service* and *Privacy Policy*, and click **Continue**. 5. Choose a pricing plan and set up payment, or skip this step and subscribe later. After creating your team, you'll be redirected to its admin console. A default project is automatically created. ## Step 2. Associate your user ID with a subscription 1. From the top navigation bar, select **Subscriptions & plans**. 2. In the target subscription row, click the value in the **Associated with** column. 3. In the dialog box that is displayed, select your user ID and click **Submit**. ## (Optional) Step 3. Get your API key of the target project If you want to use Powerdrill Enterprise Open API, this step is mandatory. 1. Choose **Projects** from the top navigation bar. 2. Click the **Default** project info card. 3. On the **Users** tab that is displayed by default, click **Add user**. 4. In the dialog box that is displayed, select your user ID and click **Add**. 5. Click the **Project API keys** tab and click the **+ API access key** button. 6. In the dialog box that is displayed, set a name and click **Create**. 7. Copy and save your API key properly and click **I have saved my secret key** to finish the creation. ## Step 4. Use Powerdrill Enterprise to run data jobs To run data jobs on your personal space: 1. Click your profile icon on the upper-right corner of the page and click **Switch Workspace**. 2. On the page that appears, click the **My personal space** card. 3. Upload your data and start exploring it. To call Open API, refer to [API Reference](/api-reference/v2/overview.mdx). # Team Settings Source: https://docs.powerdrill.ai/enterprise/settings The Settings page allows you to modify your team info On the **Settings** page of the Admin console, you can update your organization information, change payment methods, and more. *** ## Manage general settings On the **Settings** tab, there are three sections: * **Basic information** * **Payment settings** * **DANGER ZONE** In the **Basic information** section, you can update your organization name and view the admin's email account. You can only view the admin account; it cannot be changed. In the **Payment settings** section, you can add a credit card as your payment method if one hasn't been configured yet. You can also update your payment method here. To change a credit card, perform the following steps: 1. Click **Change credit card**. 2. In the dialog box that is displayed, enter your new credit card, and click **Change credit card.** In the **DANGER ZONE** section, you have the option to delete your organization. **This action is highly discouraged**—once deleted, all assets in your team, including datasets and job execution history, will be permanently erased and cannot be recovered under any circumstances. Proceed with extreme caution. *** ## The Team API keys tab The **Team API keys** tab allows you to manage your team API keys. These keys are used for authentication in operations related to project management, such as creating or modifying projects. When creating an API key, make sure to save the generated secret key, as it will only be displayed once on the console. If you forget or lose it, the only option is to create a new key. *** ## FAQ ### What should I do if the system prompts me to check my payment settings? If the system prompts you to check your payment settings, it means the credit card currently set for payment is unavailable. To avoid any service interruptions, please update your payment method by following these steps: 1. On the Admin console of your team, select **Settings** from the top navigation bar. 2. In the **Payment settings** section, click **Change credit card**. 3. In the dialog that appears, enter the new credit card details and click **Change credit card**. Your new credit card will replace the old one for payment. ### How can I delete my organization? Deleting your organization is a serious action and cannot be undone. We strongly recommend carefully considering whether it's absolutely necessary before proceeding. Please make sure this is the right decision for your team and data. Once you delete your organization, the following will be lost: * All users will be removed from the team. * All subscription records will be deleted. * All projects and datasets within those projects will be removed. * Credit card information will be deleted. * Usage and billing data will be erased. If you still wish to delete your organization, please follow these steps in the Admin console: 1. Cancel all subscriptions to job plans (if any): Go to the **Subscriptions & plans** page, choose **Manage** > **Unsubscribe** in the **Actions** column for each subscription, and confirm the operation. 2. Cancel your subscription to the AI Workspace capacity plan (if any): Go to the **Usage & billing** page, click **Unsubscribe** in the **Workspace capacity** section. 3. Delete your organization: Go to the **Settings** page, click **Delete organization** in the **DANGER ZONE** section, and confirm the action. ### What's the difference between team API keys and project API keys? They serve for different purposes. * **Team API keys**: Obtained from the **Settings** page, they are used for authenticating actions related to creating or modifying projects within your team. * **Project API keys**: Obtained from the **Projects** page, they are used for authenticating operations to resources within specific projects. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Subscriptions and Plans Source: https://docs.powerdrill.ai/enterprise/subscriptions Manage your subscriptions and plans for greater cost efficiency *** To enable users in your team to run jobs on Powerdrill Enterprise, you must subscribe to job plans. To increase the capacity of your AI Workspace, you need to subscribe to AI Workspace capacity plans. This guide explains how to manage your subscriptions to maximize cost efficiency. *** ## Subscribe to a plan ### Subscribe to a job plan Since users in your team can only run jobs after being associated with a pricing plan, it's important to understand the number of users in your team and their estimated job usage. If you're unsure, no worries. You can subscribe to a plan later as needed. Start with the Tier 1 plan, and when a user exceeds the job quota, you can easily upgrade their plan to meet their requirements. To subscribe a job plan, perform the following: 1. On the top navigation bar in the Admin console, select **Subscriptions & plans**. 2. On the page that appears, click **Subscribe**. 3. On the **Select plan** page, choose the subscription type, select the plan, and set the number of subscriptions you want to make, and click **Confirm**. There are two subscription types available: * **Monthly subscription**: Recurs on a monthly basis. * **One-time purchase**: Non-recurring and expires after the set period. When deciding on the number of subscriptions, consider how many users in your team need to run jobs. Ensure that the number of subscriptions matches the number of users in your team. 4. Once the payment is successful, you can manage your subscriptions on the **Subscriptions & plans** page. ### Subscribe to a Workspace capacity plan Each team receives 100 MB of free Workspace capacity upon creation. If your team's needs exceed this capacity, you can subscribe to a paid plan. 1. In the Admin console, select **Usage & billing** from the top navigation bar. 2. In the **Workspace capacity** section, click **Upgrade**. 3. On the **Select capacity plan** page, choose your desired plan and click **Upgrade**. Since your team shares the same pool of Workspace capacity (not tied to user or project), only one subscription to the target plan is needed. 4. After the payment is processed, you can manage your subscriptions on the **Subscriptions & plans** page. *** ## Manage your subscriptions ### Manage your job subscriptions You can manage your job plan subscriptions on the **Subscriptions & plans** page. Once on the **Subscriptions & plans** page, you'll be able to view all your active job plan subscriptions. Parameter description: * **Subscription ID**: The unique identifier for the subscription. * **Pricing tier**: The subscribed pricing tier. * **Effective date**: The date when the subscription becomes active. * **Associated with**: The user to whom the subscription is assigned. If marked as **None**, it indicates that the subscription is not associated with any user. You can click the Expand icon to link the subscription to the desired user. * **Status**: Whether the subscription is available. * **Actions**: Manage each subscription. Supported actions include: * **Associate**: Associate the subscription with a user. * **Unsubcribe**: Cancel the supcription. Unavailable for the free plan subscription and one-time purchase. * **Resume**: Reactivate the subscription. This option is available only if the subscription was previously canceled and has not expired. ### Manage your Workspace capacity subscription You can manage your subscription to Workspace capacity plan on the **Usage & billing** page. When your Workspace capacity is insufficient, you can: * Click **Subscribe** in the **Workspace capacity** section to choose a paid plan. * Click **Upgrade** in the **Workspace capacity** section to upgrade to a higher capacity plan. If you no longer need a paid plan, click **Unsubcribe** to switch to the free 100 MB Workspace capacity. After unsubscribing from a paid Workspace plan, no new data can be uploaded once the current subscription expires. To upload more data, you'll need to manually reduce the data in your organization to below 100 MB. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Users Source: https://docs.powerdrill.ai/enterprise/users This guide describes each type of users on Powerdrill Enterprise and how to manage them ## What is a user? A user in Powerdrill Enterprise is an individual who has access to a workspace within the Powerdrill Enterprise environment. *** ## User types Powerdrill Enterprise Edition provides three types of users, each designed with varying permissions to meet diverse requirements: * **Team member**: A team member is a user with full access to all features on the console. They collaborate with other team members, managing and interacting with data within the projects they are part of. Team members can perform the following operations in projects where they hold membership: * Create and manage datasets and data sources * Run jobs based on the datasets * Collaborate with other team members Team members are ideal for teams using Powerdrill Enterprise as their AI-driven data analytics platform, empowering each individual to optimize data analysis based on their personalized needs. Team members do not have access to the Powerdrill Open API, as it is unnecessary for their role. * **System user**: A system user can utilize the Powerdrill Open API to run jobs in large batches, but does not have access to the memory feature. System users can perform the following operations in projects where they hold membership: * Leverage Powerdrill Enteprise's AI capabilties via API calls. * Run jobs in batches efficienctly. * Develop custom solutions via the Powerdrill Open API to serve their users. System users are ideal for your organization's developers, data engineers, and technical personnel who need to automate tasks, run jobs in large batches, or integrate Powerdrill's AI capabilities into their custom applications. They are best suited for scenarios requiring API-driven operations or secondary development to extend services to end-users. * **Virtual users**: A virtual user can use the Powerdrill Open API to run jobs, with access to the memory feature. Virtual users are best suited for external customers or clients of your organization who need controlled access to specific features of Powerdrill. They are ideal for scenarios where users require API-based functionality along with access to the memory feature. Virtual users ensure a secure and tailored experience without exposing unnecessary details about Powerdrill. *** ## Create a user Perform the following steps to invite users to join in your project: 1. Sign in to Powerdrill. 2. Click your profile icon in the lower-left corner. 3. In the menu, go to the **Workspace** section and click **Switch Workspace**. A list of all the workspaces you're in will appear. 4. Click the **Admin console** next to the team that you own. 5. On the top-navigation bar, select **Users**. On the **Users** page, click **Create user**. 6. On the **Create user** page, select the user type you want to create, configure user information, and click **Create**. For details about how to select the user type, see [User types](#user-types). Parameter description: | Parameter | Required | Description | | :-------------- | :------- | :----------------------------------------------------------------------------------------------------------------------- | | Email | Yes | The email address of the invitee. | | Associate plan | No | The subscription ID the user will be associated with. Only users associated with a plan can run jobs. | | Add to projects | No | Select the projects you want the user to join. Users can perform operations only in projects where they hold membership. | Parameter description: | Parameter | Required | Description | | :-------------- | :------- | :----------------------------------------------------------------------------------------------------------------------- | | Display name | Yes | The identifier that represents the user publicly. | | Associate plan | No | The subscription ID the user will be associated with. Only users associated with a plan can run jobs. | | Add to projects | No | Select the projects you want the user to join. Users can perform operations only in projects where they hold membership. | Parameter description: | Parameter | Required | Description | | :-------------- | :------- | :----------------------------------------------------------------------------------------------------------------------- | | Display name | Yes | The identifier that represents the user publicly. | | Associate plan | No | The subscription ID the user will be associated with. Only users associated with a plan can run jobs. | | Add to projects | No | Select the projects you want the user to join. Users can perform operations only in projects where they hold membership. | Once the user is created, you can view their details on the **Users** page, including the user ID, email or display name, user type, project memberships, and associated plan. If you create a team member, you can view the user information only after the invitee accepts your invitation. *** ## Check user information You can check information about each user existed in your team. In this list, you can check the following information: | Field | Description | | :------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | User ID | The ID and username/display name of the user are displayed. The user ID serves as the unique identifier for a user within your team and is prefixed with **tmm-**. If the user is a **team member**, their username is shown. For **system or virtual users**, their display name is shown instead. | | User type | The user type, which can be one of the following: **Team member**, **System user**, or **Virtual user**. For details on the differences, refer to [User types](#user-types). | | Project memberships | A list of projects that the user is in. | | Associated plan | The pricing tier and its corresponding subscription ID associated with the user. The subscription ID is prefixed with **ts-**. | | Actions | Actions that can be performed on the user. | *** ## Manage project memberships for a user On the **Users** page, you can add a user to or remove a user from projects. Users not added to any projects will face the following limitations when using Powerdrill Enterprise: * Unable to upload data sources or create datasets. * Cannot specify datasets for question answering. Therefore, we highly recommend you to add users to the appropriate projects to fully utilize Powerdrill Enterprise's data analysis capabilities. *** ## Manage the subscription for a user You can associate a user with a subscription to grant them job credits for processing data tasks on Powerdrill. You can also unassociate the subscription from the user to prevent them from consuming the job credits provided by the subscription. Only users linked to a subscription can run jobs in Powerdrill Enterprise. To know more about managing your subscriptions, see [Subscriptions](/enterprise/subscriptions). 1. On the **Users** page, locate the target user in the list and click the Expand icon in the **Associated Plan** column. 2. In the dialog box that appears, select the pricing tier and the ID of an unassigned subscription, then click **Confirm**. To unassociate a subscription from a user, simply deselect the assigned subscription and click **Confirm**. *** ## Delete a user from your team You can remove unnecessary users from your team. Once a user is removed, their associated subscription will be released and can be reassigned to another user. Once a user is removed, all data associated with them will be deleted. This action cannot be undone. Proceed with caution. 1. On the **Users** page, locate the unnecesary user, and click the Delete icon in the **Actions** column. 2. In the confirmation dialog box, click **Yes, revoke**. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # What Is Job? Source: https://docs.powerdrill.ai/enterprise/what-is-job Jobs are data tasks effortlessly handled by Powerdrill through natural language conversations On Powerdrill, a job is the fundamental unit used to measure the work performed by the platform. Billing is also based on the number of jobs you plan to create. Typically, a job encompasses the entire process, from Powerdrill receiving a user prompt to delivering the corresponding response. *** ## A job's workflow The figure below illustrates the workflow of a job initiated through an API call. Upon initiating an API call, a job begins: 1. Identify user intent. 2. Create an execution plan for the task. 3. Execute the task, where Powerdrill interacts with the user's dataset to generate output aligned with the user's intent, utilizing LLM capabilities. ## What is considered a job? Each time you send a message to Powerdrill and receive a response, one job is deducted from your account. To maximize the value of each job, we recommend asking specific questions or clearly stating what you want to know or get from your data. This ensures that every job is used effectively and delivers meaningful results. Here are some examples: A good one: ``` How do "Digital Services for Taxpayers" scores vary across economies? ``` In this scerario, Powerdrill will analyze your data and provide specific answers and insights tailored to your question. A bad case: ``` Hello, Powerdrill. ``` In this case, Powerdrill will simply greet you back without delivering any meaningful information, but it will still consume 1 job from your account. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Workspaces and Teams Source: https://docs.powerdrill.ai/enterprise/workspaces This guide explains the concepts of workspaces and teams, and provides instructions on how to manage them ## Key terms ### Workspace A workspace is a dedicated Powerdrill environment with its own settings, members, and resources. Users can access multiple workspaces, such as a personal workspace and a Powerdrill Enterprise workspace. Each workspace is tailored to meet different needs and functions. ### Team A team is the virtual representation of a Powerdrill Enterprise workspace. **Each account can create only one team** but can be invited to join multiple teams as a user. This means an account can be the owner of just one team. *** ## Create a workspace **Personal Workspace:** When you sign in to Powerdrill, a personal workspace is automatically created for you. No additional steps are required. **Team space:** To get a team space, you need to create a team. The procedure is as follows: 1. Sign in to Powerdrill. 2. Click your profile icon in the lower-left corner. 3. In the menu, go to the **Workspace** section and click **Create team**. 4. Enter your organization name, agree to the *Terms of Service* and *Privacy Policy*, and click **Continue**. 5. Choose a pricing plan and set up payment, or skip this step and subscribe later. Your team is now ready to manage. An account can create only one team, but you can be a regular user in other teams if added or invited by their admins. *** ## Switch workspace To switch between workspaces, perform the following steps: 1. Sign in to Powerdrill. 2. Click your profile icon in the lower-left corner. 3. In the menu, go to the **Workspace** section and click **Switch Workspace**. A list of all the workspaces you're in will appear. 4. From here, you can either click the **Admin console** to access your team's management console or select the project you want to enter. *** ## Delete a workspace ### Delete your team space You can only delete the team you created. Deleting a workspace permanently removes the entire organization. This action is irreversible and cannot be undone. We strongly recommend carefully evaluating whether this step is absolutely necessary. Please ensure this decision is the right choice for your team and data. Once you delete your organization, the following will be lost: * All users will be removed from the team. * All subscription records will be deleted. * All projects and datasets within those projects will be removed. * Credit card information will be deleted. * Usage and billing data will be erased. If you still wish to delete your organization, please follow these steps in the Admin console: Don't know how to enter the Admin console? Refer to this [FAQ](/enterprise/enter-admin-console.mdx). 1. Cancel all subscriptions to job plans (if any): Go to the **Subscriptions & plans** page, choose **Manage** > **Unsubscribe** in the **Actions** column for each subscription, and confirm the operation. 2. Cancel your subscription to the AI Workspace capacity plan (if any): Go to the **Usage & billing** page, click **Unsubscribe** in the **Workspace capacity** section. 3. Delete your organization: Go to the **Settings** page, click **Delete organization** in the **DANGER ZONE** section, and confirm the action. ### Delete your private workspace Deleting your private workspace is the same as deleting your account from Powerdrill. If your account is the admin of a Powerdrill Enterprise team, refer to [How Can I Delete My Account After I Have Created a Team on Powerdrill Enterprise?](/enterprise/delete-account) for a detailed guide. If your account does not belong to any team, simply follow \[Step 3]\(/enterprise/delete-account#step-3- delete-your-account) to delete your account. *** ## FAQ ### How many teams can I create using the same account? One account can only be the owner of up to one team. This means, you can create a team only when there is no team is owned by your account. However, you can be regular users in multiple teams as long as your account has been added to other teams. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Advanced Analytics Source: https://docs.powerdrill.ai/features/advanced-analytics Gives full play to the value of your data ## Introduction Are you finding yourself overwhelmed with raw data, struggling to discover the insights you need? Are your decisions based more on instinct rather than proves? Your solution is right here -- Powerdrill launches its new feature **Advanced Analytics**. Advanced Analytics can generate visualizations such as charts and maps after analyzing raw data. With Advanced Analytics, all you need to do is to "tell" Advanced Analytics what you want in natural language and let it uncover the trends and patterns in your data. We are excited to announce our extension on this feature -- SQL Advanced Analytics. This powerful new tool allows you to seamlessly integrate your SQL databases as data sources. Once connected, Powerdrill empowers you to delve into your data with the full suite of SQL commands, unlocking sophisticated analytical possibilities. *** ## How to use general Advanced Analytics 1. Sign in to [Powerdrill](https://powerdrill.ai). 2. Select **Advanced Analytics** and select the files you want to analyze. Only CSV, TSV, and Excel files are supported. Up to 10 files can be uploaded at a time. 3. Chat with Powerdrill to start your conversation over your dataset. If you just want to see the effect of Advanced Analytics before uploading your own data, you can try our demo to take a glimpse. *** ## How to use SQL Advanced Analytics To use SQL Advanced Analytics, you must first connect your SQL databases to Powerdrill. Currently, Powerdrill supports two types of SQL databases: MySQL and PostgreSQL. At Powerdrill, we prioritize the security of your data and privacy above all else. We have implemented robust security measures to ensure that your keys and passwords are never accessed or stored by our system. 1. Sign in to [Powerdrill](https://powerdrill.ai). 2. In the left sidebar, select **Datasets**. 3. On the page that is displayed, click **+ Dataset** in the upper-right corner. 4. On the **Create Dataset** page, click the **SQL Databases** tab. If this is your first time using this feature, the platform will prompt you to specify a user with read-only access to your database. For details about how to create a user with read-only privileges on the database, see [Set user privileges to read-only](#set-user-privileges-to-read-only). 5. Select your SQL database type. * For PostgreSQL-compatible databases, select **PostgreSQL**. * For MySQL-compatible databases, select **MySQL**. 6. Configure credentials. * If you choose the `General` form, you need to specify the hostname, port, and name of the database to connect. * If you choose the `Advanced` form, fill in the domain name of the database to connect to complete the URL. 7. Configure authentication information, including the username and password used to connect to the database. 8. If the database is configured with IP allowlists or blocklists, ensure the IP addresses provided on the page are in the allowlists. 9. Click **Test Connection**. If the connection is successful, click **Finish**. Now, you can ask questions about anything you want to know from your database. ### Set user privileges to read-only The detailed procedure varies with your database type: 1. Connect to your PostgreSQL database as a superuser or a user with the necessary privileges to create roles and assign permissions. For example, run the following command in the `psql` CLI: ```bash theme={null} psql -U postgres ``` Replace `postgres` with your superuser or administrative username. 2. Run `CREATE ROLE` to create a role. ```sql theme={null} CREATE ROLE readonly_user WITH LOGIN PASSWORD 'password'; ``` Replace `readonly_user` with the desired username and `password` with a strong password. Alternatively, you can run `CREATE USER` as a shortcut for `CREATE ROLE ... WITH LOGIN`. 3. Grant the `CONNECT` privilege on the database to the user. ```sql theme={null} GRANT CONNECT ON DATABASE TO readonly_user; ``` 4. Grant the `USAGE` privilege on the schema where your data resides. ```sql theme={null} GRANT USAGE ON SCHEMA TO readonly_user; ``` 5. Grant the `SELECT` privilege on the tables in the schema. ```sql theme={null} GRANT SELECT ON ALL TABLES IN SCHEMA TO readonly_user; ``` 6. Set default privileges to allow `readonly_user` has the `SELECT` privilege on tables future created in the schema. ```sql theme={null} ALTER DEFAULT PRIVILEGES IN SCHEMA GRANT SELECT ON TABLES TO readonly_user; ``` 7. Check whether the user privileges are configured as expected in `psql`: ```bash theme={null} psql -U readonly_user -d database_name ``` 1. Use a MySQL CLI to connect to your MySQL server as the root user or another user with sufficient privileges to create new users and grant permissions. Following uses the `root` user and the `mysql` CLI as an example. ```bash theme={null} mysql -u root -p ``` 2. Enter the password as prompted. 3. Create a user and set a password for the user. ```sql theme={null} CREATE USER 'readonly_user'@'localhost' IDENTIFIED BY ''; ``` * `readonly_user` is an example only. Replace it as needed. * If you want to allow the user to connect from any host, replace `localhost` with `%`. 4. Grant the `SELECT` privilege on all tables in the database to the `readonly_user` user. ```sql theme={null} GRANT SELECT ON .* TO 'readonly_user'@'localhost'; ``` 5. Make the privileges take effect. ```sql theme={null} FLUSH PRIVILEGES; ``` 6. Check whether the user privileges are configured as expected in `mysql`: ```sql theme={null} mysql -u readonly_user -p ``` *** ## Need more help? Get answers from our members Tell us more and we'll help you out # AI Report Generator Source: https://docs.powerdrill.ai/features/ai-data-reports Generate high-quality data report in one click ## Introduction Powerdrill's AI Report Generator is a newly released feature designed to accelerate data insight discovery. By harnessing AI, it streamlines the creation of detailed, comprehensive reports, significantly reducing the time and effort required. Whether you're in healthcare, finance, marketing, or other industries, this feature enhances efficiency by quickly uncovering key insights hidden within your data. Transforming the way you process information, Powerdrill's AI Data Report Generator enables faster, more accurate data-driven decision-making. With the AI Report Generator, turning raw data into a comprehensive report is just one click away. This powerful feature allows users to streamline data analysis and reporting, unlocking the full potential of your data with minimal effort. Key benefits include: * **Automatic insights**: Effortlessly uncover valuable insights from your raw data, giving you a clearer understanding of the underlying information. * **One-click generation**: The process is as simple as it gets—just click a button and let Powerdrill take over. In no time, your raw data will be transformed into a polished, organized report. * **Automated data visualization**: Instantly convert data insights into visually appealing charts and graphs. * **PowerPoint conversion**: Need to present your data? No problem. With one click, your report is converted into ready-to-use PowerPoint slides, helping you create impactful presentations without any hassle. * **Customized content**: Don't like something in your report? Simply edit the question to fit your needs, and Powerdrill will instantly generate fresh content based on your new query. * **Easy sharing**: Sharing your insights with others is effortless. You can share the link to your reports with your teammates, friends, or anyone you want, or download your reports and visualizations as PNG or PDF files. ## How to use AI Report Generator ### Method 1. Generate a report by uploading new files 1. In the AI data agent section, find **AI Data Report Generator**, and click **Get started**. Alternatively, click **Try our demo** if you don't have a suitable data file at hand. 2. Select the files you want to analyze. Up to 10 files can be uploaded at a time. If you want to analyze more files, check [method 2](#method-2-generate-a-report-from-an-existing-dataset) 3. Wait for the report to generate. Once ready, you can download the report as a Word, Markdown, or PDF file, or open it directly in Notion or Google Docs for editing and sharing. 4. Edit the questions you don't like and let Powerdrill regenerate content based on your new questions. You can also delete questions to remove related content from your report. Additionally, you can click **Convert to presentation** to transform the data report into a presentation. For more details on creating AI presentations, see [AI Presentation Maker](features/ai-presentation). ### Method 2. Generate a report from an existing dataset 1. On the homepage, click **Start a new chat** at the bottom. 2. In the upper-right corner, click **Select dataset**. 3. Select the dataset you want to use. Ensure the dataset you select contain only CSV, TSV, Excel files. Otherwise, the report cannot be generated. 4. In the chat session page, click **Generate data report** in the upper-right corner. 5. Wait for the report to generate. Once ready, you can download the report as a Word, Markdown, or PDF file, or open it directly in Notion or Google Docs for editing and sharing. 6. Edit the questions you don't like and let Powerdrill regenerate content based on your new questions. You can also delete questions to remove related content from your report. Additionally, you can click **Convert to presentation** to transform the data report into a presentation. For more details on creating AI presentations, see [AI Presentation Maker](features/ai-presentation). *** ## FAQ ### What types of data are supported by the AI Report Generator? The AI Report Generator currently supports CSV, TSV, and Excel file formats. ### Can I customize the charts/graphs generated in the data report? Yes, some charts and graphs are customizable. For those that support customization, you'll see a toolbar in the upper-left corner of the chart, like this: You can change the graph type. The AI Report Generator currently supports Line, Area, Bar, Pie, Doughnut, and Table graphs. Switch to the type that best fits your needs. Additionally, you can download these charts/graphs as PNG or CSV files for further use. ## FAQ ### Can I edit the generated data report online? No, but you can download the data report to your device or save it directly to your Notion or Google Docs account for further editing. ### Can I specify queries contained in a data report? No, the data report is generated automatically. However, if you'd like to add personalized insights, you can follow these steps: 1. Download the data report or save it to your Notion or Google Docs account. 2. Start a new chat. 3. In the **Data Insights** tab, click the **Select dataset** button, and find the dataset that contains the files you used to generate the data report. If you cannot find the dataset, you can upload the files again to create a new dataset instead. 4. Type your query in the message box, and let Powerdrill provide insights. 5. Once Powerdrill generates the insights, you can copy the content (both visuals and text) into your data report. ### Is this feature free? Free plan users can generate one data report per month. To generate more reports, you can [upgrade your plan](https://powerdrill.ai/pricing). *** ## Need more help? Get answers from our members Tell us more and we'll help you out # AI Presentation Maker Source: https://docs.powerdrill.ai/features/ai-presentation Transforms your files into clear, organized presentations, reducing hours of work to minutes ## Introduction Creating presentations can often feel tedious and time-consuming, but with Powerdrill's AI Presentation Maker, you can streamline the process and complete your slides in no time. Whether you're preparing a business pitch, a school project, or a client presentation, this tool simplifies the task, making it both quick and efficient. The new feature **AI Presentation Maker**, which generates clear, organized slide decks based on the files you upload, provides the following key benefits: * **Automatic creation of tailored presentations**: Simply upload your files, click through a few steps, and within minutes, Powerdrill will generate a complete presentation for you. * **Beautiful templates for various scenarios**: Powerdrill offers a curated collection of high-end templates designed to meet different needs. Whether you're preparing for a meeting, workshop, or lecture, you'll find a template that suits your requirements. * **Time-saving**: By automating the whole process, AI Presentation Maker helps you save both time and money. This allows you to focus on the delivery of your presentation. *** ## How to use the AI Presentation Maker 1. Sign in to [Powerdrill](https://powerdrill.ai). 2. From the left sidebar, choose **Presentations**. You will be directed to a page similar to this: Alternatively, you can start directly from the home page: 3. Then, upload your file. For CSV/TSV/Excel files, you can upload up to 10 files at a time. 4. Now, all you need to do is waiting for the completion. This process takes just one or two minutes. After the presentation is generated, you can download it as PowerPoint slides and customize the content locally. All generated presentations are stored on the **AI presentations** page. You can access them anytime by selecting **Presentations** from the leftside bar. *** ## FAQ ### What file types are supported by the AI Presentation Maker? Are there any restrictions? You can upload the following file types for your presentation: * **PDF:** Only one PDF file can be uploaded per presentation. * **Word:** Only one Word document can be uploaded per presentation. * **CSV/TSV/Excel:** You can upload up to 10 files, which can be a combination of CSV, TSV, or Excel formats. ### Can I download the generated presentation as PowerPoint slides? Yes, you can download the presentation as PowerPoint slides. ### Will the downloaded slides include watermarks? It depends on your subscription plan. Free users can generate and download one presentation, but it will include watermarks. If you upgrade your plan, the downloaded slides will be watermark-free. ### Can I edit the presentation online? Currently, Powerdrill does not support online editing. However, you can download the slides and edit them locally on your device. ### How can I control the quality of the generated presentation? The quality of the generated presentation is directly based on the files you upload. For the best results, ensure that you upload only relevant information, so that Powerdrill can accurately process the data and create high-quality slides. *** ## Need more help? Get answers from our members Tell us more and we'll help you out # Chat App over Dataset Source: https://docs.powerdrill.ai/features/chat-app Delivers smarter and more accurate answers to questions based on your datasets than ChatGPT ## Introduction ChatGPT has already shown exciting capabilities in understanding natural language and applying public internet knowledge. However, there is still room for improvement in its deep understanding of specific domains and private data. Fortunately, we can guide ChatGPT to better understand and answer questions using appropriate prompts. Furthermore, we can fine-tune OpenAI's LLMs for specific domains and private data to enhance its understanding of domain knowledge and private data. Building on this, Powerdrill has integrated several public Apps that have been tailored to specific datasets. These Apps have undergone **knowledge processing** and **knowledge indexing**, so they can answer questions about the specific domains and data more accurately than ChatGPT with general LLMs. This unique feature of Powerdrill provides a more precise and efficient solution for domain-specific inquiries, thereby bridging the gap between your data and AI. With Powerdrill, you can leverage the power of AI to transform your data into a valuable knowledge base. This surpasses the accuracy of ChatGPT and provides a solution better tailored to your specific needs. *** ## How to use this feature Simple Chat — a built-in chat App — is used as an example App in the following procedure to describe how to use Chat App over Dataset feature. ### Prerequisites You have signed in to Powerdrill and created a dataset with Simple Chat associated. For more information about how to log in to Powerdrill and create a dataset, see Steps 1 and 2 in [Try Out Powerdrill in Minutes](/quick-start#step-1-sign-in-to-powerdrill). ### Procedure Method 1: 1. On the **Datasets** page, find the row where the target dataset resides, click **Simple Chat** in the **Associated Apps** column. If **Multiple Apps** is displayed in the **Associated Apps** column, click the button next **Multiple Apps** and select Simple Chat from the Apps list. 2. In the new session, ask questions based on your dataset. Method 2: 1. On the **Apps** page, click **Simple Chat**. 2. In the upper-right corner of the new session, select your dataset in the **Associate Dataset** drop-down list. 3. Ask questions based on your dataset. *** ## Watch a demo