Forms

Learn how to create, manage, and analyze form definitions on the Weav.ai platform. This guide covers creating forms, updating definitions, filtering instances, and running analytics for efficient document processing.

Overview

The Forms section enables users to define, manage, and analyze form structures on the Weav.ai platform. By setting up form definitions, users can extract structured data, run analytics, and streamline workflows with customizable fields and parameters. This guide provides detailed instructions for creating forms, filtering instances, and executing analytics on extracted data.

Prerequisite - To get started, ensure your environment is properly configured by following the Setup Guide.

Create form

Creating a form definition. A form definition is required before running process_form workflow.

python3 documents/forms/create_form.py --name "new form" --category "new" --description "test" --is_shared true --is_searchable true --fields "[{\n  \"name\": \"MICROSOFT FORM\",\n  \"description\": \"A form for microsoft\",\n  \"category\": \"ANNUAL REPORT\",\n  \"fields\": [\n    {\n      \"name\": \"Cost of revenue\",\n      \"field_type\": \"Number\",\n      \"is_array\": false,\n      \"fill_by_search\": false,\n      \"description\": \"Extract cost of revenue\"\n    }\n  ],\n  \"is_searchable\": false,\n  \"is_shared\": false\n}]"

Parameters:

Parameter	Description	Required/Optional	Allowed values
name	The form name	Required
category	A category name for the form	Required
description	The description of the form	Required
is_shared	A flag to decide sharing permissions	Optional (default : False)	false, f, False, true, t, True
is_searchable	A flag to decide visibility	Optional (default : False)	false, f, False, true, t, True
fields	Form fields	Optional	Stringified JSON

Format for fields

[{
  "name": "str",
  "description": "str",
  "category": "str",
  "fields": [
    {
      "name": "str",
      "field_type": "Number",
      "is_array": boolean,
      "fill_by_search": boolean,
      "description": "str"
    }
  ],
  "is_searchable": boolean,
  "is_shared": boolean
}]

## Example stringified version for CLI

"[{\n  \"name\": \"MICROSOFT FORM\",\n  \"description\": \"A form for microsoft\",\n  \"category\": \"ANNUAL REPORT\",\n  \"fields\": [\n    {\n      \"name\": \"Cost of revenue\",\n      \"field_type\": \"Number\",\n      \"is_array\": false,\n      \"fill_by_search\": false,\n      \"description\": \"Extract cost of revenue\"\n    }\n  ],\n  \"is_searchable\": false,\n  \"is_shared\": false\n}]"

Key	Description	Required/Optional	Allowed values
name	Name of the Entity	Required
field_type	The data type of the entity	Required	“Number”, “Date”, “Text”, “Table”
description	A short instruction to the prompt about the field	Optional
is_array	If it’s an entity with multiple values	Optional (default: False)	True, False
fill_by_search	Use internet search to fill this information	Optional (default: False	True, False

Response:

{
   "category":"new",
   "created_at":datetime.datetime(2024, 9, 25, 20, 12, 11, tzinfo=datetime.timezone.utc),
   "description":"True",
   "fields":[
      {
         "description":"Net Sales for the quarter",
         "field_type":"Number",
         "fill_by_search":false,
         "is_array":true,
         "name":"Net Sales"
      }
   ],
   "id":"66f46e9b70dd6d497d9b8a37",
   "is_searchable":true,
   "is_shared":true,
   "name":"new form",
   "user_id":"google-oauth2|117349365869611297391"
}

Delete Form definition

Deleting a form definition. The user is prompted to reconfirm the deletion.

python3 documents/forms/delete_form_definition.py --form_id 66ea66d547fff0950cba17e

Parameters:

Parameter	Description	Required/Optional
form_id	The unique identifier of the form	Required

Response:

{
   "category":"new",
   "created_at":datetime.datetime(2024, 9, 25, 20, 12, 11, tzinfo=datetime.timezone.utc),
   "description":"True",
   "fields":[
      {
         "description":"Net Sales for the quarter",
         "field_type":"Number",
         "fill_by_search":false,
         "is_array":true,
         "name":"Net Sales"
      }
   ],
   "id":"66f46e9b70dd6d497d9b8a37",
   "is_searchable":true,
   "is_shared":true,
   "name":"new form",
   "user_id":"google-oauth2|117349365869611297391"
}

Search capabilities to retrieve form definitions

python3 documents/forms/filter_form.py --query "SECURITIES AND EXCHANGE COMMISSION" --scope "all_forms"

Parameters:

Parameter	Description	Required/Optional	Allowed values
scope	The scope of search	Required	all_forms, my_forms
is_searchable	Filter for visibility	Optional (default : False)	false, f, False, true, t, True
query	When applied, string matches category	Optional (default : “”)

Response:

{
   "forms":[
      {
         "category":"SECURITIES AND EXCHANGE COMMISSION",
         "created_at":"2024-09-25T09:13:50Z",
         "description":"",
         "fields":[
            {
               "description":"",
               "field_type":"Date",
               "fill_by_search":false,
               "is_array":false,
               "name":"testEnitity1"
            }
         ],
         "id":"66f3d44eeb87303bc52bb9b4",
         "is_searchable":false,
         "is_shared":false,
         "name":"test",
         "user_id":"google-oauth2|117349365869611297391"
      }
   ]
}

Get form definitions

Fetching the form definition of a particular form.

python3 documents/forms/get_form_definition.py --form_id "66f46e9b70dd6d497d9b8a37

Parameters:

Parameter	Description	Required/Optional
form_id	Form ID	Required

Response:

{
   "category":"SECURITIES AND EXCHANGE COMMISSION",
   "created_at":"2024-09-25T09:13:50Z",
   "description":"",
   "fields":[
      {
         "description":"",
         "field_type":"Date",
         "fill_by_search":false,
         "is_array":false,
         "name":"testEnitity1"
      }
   ],
   "id":"66f3d44eeb87303bc52bb9b4",
   "is_searchable":false,
   "is_shared":false,
   "name":"test",
   "user_id":"google-oauth2|117349365869611297391"
}

Update form definition

Updating the definition of a form such as name, description, entities and their data etc.

python3 documents/forms/update_form_definition.py --form_id 66f46e9b70dd6d497d9b8a37 --name "update" --category "new" --description "Test desc" --is_shared false --is_searchable false

Parameters:

Parameter	Description	Required/Optional	Allowed values
form_id	Form identifier	Required
name	Form name	Required
category	Form category	Required
description	Form description	Optional (default : “”)
is_shared	Filter for sharing permissions	Optional (default : False)	false, f, False, true, t, True
is_searchable	Filter for visibility	Optional (default : False)	false, f, False, true, t, True
fields	Form fields	Optional	Stringified JSON

Format for fields

[{
  "name": "str",
  "description": "str",
  "category": "str",
  "fields": [
    {
      "name": "str",
      "field_type": "Number",
      "is_array": boolean,
      "fill_by_search": boolean,
      "description": "str"
    }
  ],
  "is_searchable": boolean,
  "is_shared": boolean
}]

## Example stringified version for CLI

"[{\n  \"name\": \"MICROSOFT FORM\",\n  \"description\": \"A form for microsoft\",\n  \"category\": \"ANNUAL REPORT\",\n  \"fields\": [\n    {\n      \"name\": \"Cost of revenue\",\n      \"field_type\": \"Number\",\n      \"is_array\": false,\n      \"fill_by_search\": false,\n      \"description\": \"Extract cost of revenue\"\n    }\n  ],\n  \"is_searchable\": false,\n  \"is_shared\": false\n}]"

Key	Description	Required/Optional	Allowed values
name	Name of the Entity	Required
field_type	The data type of the entity	Required	“Number”, “Date”, “Text”, “Table”
description	A short instruction to the prompt about the field	Optional
is_array	If it’s an entity with multiple values	Optional (default: False)	True, False
fill_by_search	Use internet search to fill this information	Optional (default: False	True, False

Response:

{
   "category":"SECURITIES AND EXCHANGE COMMISSION",
   "created_at":"2024-09-25T09:13:50Z",
   "description":"",
   "fields":[
      {
         "description":"",
         "field_type":"Date",
         "fill_by_search":false,
         "is_array":false,
         "name":"testEnitity1"
      }
   ],
   "id":"66f3d44eeb87303bc52bb9b4",
   "is_searchable":false,
   "is_shared":false,
   "name":"test",
   "user_id":"google-oauth2|117349365869611297391"
}

A search query for retrieving form instances.

python3 documents/forms/filter_form_instances.py --scope all_documents --status "DONE" --category "SECURITIES AND EXCHANGE COMMISSION"

Parameters:

Response:

{
   "total":4,
   "form_instances":[
      {
         "form_instance":{
            "data":[
               {
                  "name":"testEnitity1",
                  "value":"2023-08-03T00:00:00",
                  "identifier":"1f929c08-655e-4cf2-845c-18e3eb428ce7",
                  "weav_page_number":24
               }
            ],
            "metadata":{
               "modified_at":"2024-09-25T22:09:52.016000",
               "status":"DONE"
            }
         },
         "doc_id":"66e0fba3089fbd21c4dd80c3",
         "form_id":"66f3d44eeb87303bc52bb9b4",
         "file_name":"AAPL_10Q.pdf",
         "status":"AI_READY",
         "category":"SECURITIES AND EXCHANGE COMMISSION",
         "in_folders":[
            "66e0f93093798ee1c937e39a"
         ],
         "owner_id":"google-oauth2|117349365869611297391"
      },
      .
      .
      .
      .
   ]
}

Execute form analytics

Running analytics on a form.

python3 documents/forms/execute_form_analytics.py --form_id 66f3d44eeb87303bc52bb9b4 --skip 0 --limit 25

Parameters:

Parameter	Description	Required/Optional
skip	Number of documents to skip	Optional (default 0)
limit	Total number of documents to consider	Optional (default 25)
form_id	Form Identifier	Required
query	This is a mongo pipeline query that is fetched from the Search service. (default: “{\n ‘reason_for_no_pymongo_pipeline’: ‘No user request provided’\n}”)	Optional

Response:

{
   "columns":[
      "Net Sales",
      "Total sales"
   ],
   "results":[
      {
         "Net Sales":[
            81797.0,
            82959.0,
            .
            .
            .
            .
         ],
         "metadata":{
            "_id":"66fe1b65b1d0dfb13c9975f0",
            "file_name":"AAPL_10Q.pdf",
            "in_folders":[
               
            ]
         }
      },
      {
         "Net Sales":[
            81797.0,
            82959.0,
            .
            .
            .
            .
         ],
         "Total sales":94569.0,
         "metadata":{
            "_id":"66fe29e1eb87303bc52bba93",
            "file_name":"AAPL_10Q.pdf",
            "in_folders":[
               
            ]
         }
      }
   ],
   "summary":"",
   "total_count":2
}

Download query result

python3 documents/forms/download_query_result.py --form_id 66f3d44eeb87303bc52bb9b4 --download_format "JSON"

Parameters:

Parameter	Description	Required/Optional	Allowed values
form_id	Form Identifier	Required
query	This is a mongo pipeline query that is fetched from the Search service. (default: “{\n ‘reason_for_no_pymongo_pipeline’: ‘No user request provided’\n}”)	Optional
download_format	The format in which the results need to be viewed	Optional (default : “JSON”)	JSON, CSV

Response:

--download_format = "JSON"

{'docs': [{'testEnitity1': '2023-08-03T00:00:00'}]}

--download_format = "CSV"

          testEnitity1
0  2023-08-03T00:00:00

Forms

Overview

Create form

Delete Form definition

Filter form

Get form definitions

Update form definition

Filter form instances

Execute form analytics

Download query result