Using Data Uploader

1. Ensure Requirements are met. Requirements can be found here. If using SQL on premise, follow these requirements.

2. In the Welcome to Data Uploader Setup Wizard dialog, click Next.
3. If you would like to install the Data Uploader somewhere other than the default location displayed in the installer, click Browse and navigate to your desired installation folder; otherwise, leave the path unchanged. To install the uploader for your access only, click Just me or select Everyone to allow access to anyone using your system. Click Next to continue.

Tip: It recommended to install the Data Uploader on the SQL server where the firm Time & Billing system/NBI application is being run.

4. Click Close on the Installation Complete pop-up box once it notes DataUploader has been successfully installed.

5. The Cloud Data Uploader is installed as a shortcut on your desktop. Double click to launch the installation.

Tip: The latest version of the Data Uploader is always available in CAM. For subsequent upgrades, the Data Installer must be uninstalled and installed again.

Warning: You have to complete the External System Configuration set-up to use the Cloud Data Uploader.

When the data uploader is accessed for the first time, the CAM Credentials tab on the Cloud Data Uploader window displays. The Cloud Data Uploader requires domain authentication to secure information from the source destination to the target destination.

Type information in the provided fields, based on the table below, then click Authenticate. This is a one-time process only and need not be done for subsequent updates. Re-authentication may be required in case you change your credentials or your access token fails.

Field	Description
Domain	Enter the domain name that contains the data to be synced to cloud. E.g. sitename.tenantname.io. Leave off the https:// and the ending /
Email	Enter your domain email ID. E.g. myemail@company.com
Password	Enter password for the email specified above that is used to log into CAM.

Upon successful authentication you will be brought into the Data Uploader page to the Run tab by default. Go to Connection String tab to begin.

Use the following instructions to add a new connection string to connect to the billing database. This will allow the Data Uploader to connect to the SQL Server and execute the queries to upload jobs to CAM. If you run Data Uploader without a connection string, it will return an error.

Type information in the provided fields, based on the table below and click Save. Click Test Connection, to test the connection. If successful, a Valid Connection message is displayed.

Field	Description
SQL Server Name	The name of the SQL server hosting the billing database.
Connected System	Shows the connected systems that can be set. Options are Local Sync Database, Source Database, and CAM Local Database. Local Sync is one used for DataSync. Source Database is the external system typically used. CAM local database allows connecting to a CAM local database to use.
Database	The name of the database that you created on the SQL server to be used by Data Uploader.
Authentication	Choose the type of authentication to be used to connect to the data source database. Options include Windows authentication using the Windows credentials and SQL authentication using specific database-only credentials.
Username	Enter the username of the database. (SQL authentication only)
Password	Enter the password for the username of the database. (SQL authentication only)

Then continue to the SQL Files tab, to set the sql scripts to run for the uploader process.

This tab helps to schedule the sync CAM data to local database.

Type information in the provided fields, based on the table below and click Save.

Field	Description
Select Connection String	It will list Connection String that are set in Connection String tabs where connected source is set as 'Local Sync Database'. Choose the connection string to create local data sync.
What do you want to sync?	Select any or all from options provided : Users Groups & Group Membership Matter(Workspace) Metadata
How often do you want to sync?	Select any one interval options and occurrences.   Option Availability Minutes The job is executed based on the interval in minutes to execute. Hourly The job is executed based on the time interval defined in hour(s). Daily The job is executed daily based on the time interval defined in day(s). Weekly The job is executed weekly based on the time interval defined in week(s)and the days of the week selected. Never If Never is selected, no options appear below. Click Run Now for this option when ready to run the job.

The SQL Files tab displays the default Data Uploader program folder C:\Program Files(x86)\Prosperoware\DataUploader\sql (this will point to the location where the uploader is installed). Click the link to auto create a SQL folder in this location. This SQL folder will contain all your SQL queries to be executed by the Data Uploader. The Data Uploader will access this folder and execute queries saved here against your on premise database defined in the connection string.

After upgrading Data Uploader:

• Edit the Connection string for the SQL files to point to a folder in line e in the AppSettings.config.
• Then edit the individual SQL files configurations in the SourceFiles.config.

Viewing SQL Files:

• In the Cloud Data Uploader, click the Refresh button to view the updated list of files placed in the folder.
• To view or edit the script, click Edit File.

Options to Edit are:

FileName cannot be edited.

Type of the SQL file.

Select if Enabled or not Enabled. If no, the remaining fields do not display. If yes, these fields display:

Run Every: The time period to run the SQL script.

From: The initial date of the full or delta sync.

Start: The start date of the full or delta sync

• Click Run Now to execute the query.
• Actively used SQL files will be checked in the Active box. Inactive files are not checked and used.

Once the query is executed, the Data Uploader will create a CSV folder in the default installation location C:\Program Files(x86)\Prosperoware\DataUploader\csv. Each query is returned in the form of a separate CSV file, which is placed by the Data Uploader in this CSV folder. The CSV file is then uploaded to the Jobs tab for further processing. The log details of this uploaded job can be viewed in the Log tab in the Data Uploader.

Validation of SQL Files:

This feature allows for the validation of the Data Uploader SQL files. It checks the metadata if present in CAM, if the metadata mappings are present, and that the column names are defined. Additionally, if templates are set, it will check that the templates exist in CAM. Users and Groups ACL's are also confirmed to ensure the users and groups exist in CAM.

Click Validate SQL Files from the options menu in the SQL Files tab of Data Uploader.

Any portion of the Validation that fails will have a red X. The particular failed data entries will be in the Application Log of Data Uploader.

Setup Schedule button:

The schedule for the queries to be executed can be set on the Run tab. The content from your source database will be automatically updated to CAM as per the frequency defined here. You can set the option to execute the queries Minutely, Hourly, Daily, Weeklyor Never (for a manual or one time run).

The table below lists the set up available. The timer is available for each SQl file available. Set the timer and click Save. The Last Run time and the scheduled Next Run time is updated in the tab. You can also click Run Now to execute the job manually.

Timer	Description
Minutes	The job is executed based on the start time, end time, and the interval in minutes to execute.
Hourly	The job is executed based on the start time and the time interval defined in hour(s).
Daily	The job is executed daily based on the start date, time, and the time interval defined in day(s).
Weekly	The job is executed weekly based on the start date, time, the time interval defined in week(s)and the days of the week selected.
Never	If Never is selected, no options appear below. Click Run Now for this option when ready to run the job.

The Data Uploader task is also added to the Windows Task Scheduler on the machine where the Data Uploader tool resides, if scheduled for a timeframe. The schedule for the Data Uploader can also be adjusted from the Task Scheduler. The user must have an account with permissions to enable the scheduler task.

Each CSV file will be uploaded one at a time into the Jobs tab to eliminate collisions.

Validate button:

It validates data and configuration files; and provides status.

Any portion of the validation that fails will have a red X. The particular failed data entries will be in the Application Log of Data Uploader.

Run Now button:

This will popup a window and provides options to the user to decide which files to run. It allows user to select individual SQL file Or Active Directory Sync.

Select Sql Files or Active Directory or All

If SQL Files is selected, it displays a list of active SQL files with checkboxes.

Click Run, selected files will be executed.

Click the Log tab to view information on the jobs executed in the Data Uploader, including process information and the type of error. The following type of entries are displayed:

• INFO- Any information or generic notes regarding certain jobs uploaded.
• ERROR- Displays a message describing the failure. Click Details in the message to view the error details and stacktrace.
• FATAL- Displays complete description of the error

The Log has a Filter option that allows you to narrow the entries to a particular set of results.

You can also Export the content of any log to Excel. This can be helpful when contacting Support regarding an issue. Click the Refresh button to refresh the values to show the latest log entry. Click Delete to clear the log.

Re-queue a Job:

there is a new queue parameter to re-queue any old jobs. To do this: add/update the CSV parameter of the job CSV to Queue=1 and upload.

To set the Appsettings, follow the below steps.

Open up the AppSettings.config in C:\Program Files (x86)\Prosperoware\Prosperoware Data Uploader\configs file path.

a. Token Endpoint is in Line 3. This sets the api token URL.

b. ClientUI is in Line 4. This sets the s3 UI URL.

c. DocumentAnalysisServiceEndPoint is in Line 5.

d. ShowFileShare is in Line 6. Enable this to true for fileshare integration.

e. SqlFolderPath is in Line 7. If directing to a specific SQL file path to use, set this.

f. CsvFolderPath is in Line 8. If directing to a specific location csv folder, set this.

g. CsvArchiveFolderPath is in Line 9. If using a specific location for archived csv's, set this.

h. CsvDelimiter is in Line 10. Sets if comma separated or tab. For tab it is \t, comma as \c

i. DateFormat is in Line 11. Sets the specific date format to set dates as for the application.

j. ADOption is in Line 12. Sets specific AD options to use.

k. FlatternGroups goes to Line 13. Tells to flatten AD groups or to leave as is.

l. SyncGroups goes to Line 14. Allows syncing AD groups.

m. SupressScriptError goes to Line 15. Script errors will be hidden if set to true. For enhanced diagnostics, set to false.

n. ClientSettingsProvider.ServiceUri goes to Line 16.

o. CommandTimeOut goes to Line 17. Sets the timeperiod for long running commands to timeout to 600.

p. ArchiveFilesLifeCycle goes to Line 18. Contains the period of time to archive files (e.g csvs) in days.

q. RecordCountLimit goes to Line 19. Sets the maximum record count to 100, to return in a csv.

r. ThreadCount goes to Line 20. Sets the number of threads to use in parallel.

s. Debug is NEW goes to Line 21. Enable this to true for debug.

t. ExportFolderlength goes to Line 22. Set value to 200, to restrict the name of document not more than 200 chars.

u. ExportDocumentlength goes to Line 23. Set value to 200, to restrict the name of folder not more than 200 chars.

v. DomainMachineName goes to Line 24. Used for passing events and eventl logs for AD from the windows domain.

w. SyncNestedUserMembership goes to Line 25. Should nested users be synched?

x. CAMLOCALDB is NEW goes to Line 26, and is not present in the old file. Allows setting db as sql or the default sqllite.

The application configuration can be further edited from the AppSettings.config file in the Data Uploader directory.

To restore a prior configuration: first backup the appsettings.config, then upgrade Data Uploader and restore the config, or copy the configuration to another system.

Click this tab to sync connections to an Active Directory system. Click + Add and provide details in provided fields, based on the table below.

Go to Run Tabs and click Run Now to trigger the AD sync manually.

Field	Description
Domain	The domain controller (DC) name that contains the users to be synced to CAM.
Select External Sytems	Selects which external system(s) to create the AD items on. Note this will show all available external systems configured, not just those to which your license has.
User Path	Define the common name (CN) user look up path.
Sync Groups?	Select [Yes] to sync your user groups with CAM and the configured the external system.
Group Path	Define the common name (CN) group look up path.
Nested Groups?	Select [Flatten and include] to sync your child user group members with the parent group that contain them. This will remove all the sub-groups and create a flat list of users in the parent group, gathered from all the child groups contained within the parent group. Or [Skip]
Group Exclusions	Add any groups to be excluded from this domain.
Active	Select [Yes] to activate the domain else select [No].
Authentication	If the domain is added to the Trusted site zones, you can select the Trusted Login option for auto log on with current username and password or Enter windows credentials to log on to the domain.

click Save.

The Prosperoware Cloud Data Uploader allows you to sync and manage your Active Directory users /groups across configured systems. The general workflow to manage AD sync is as follows:

1. Download, install and configure the Data Uploader.
2. Click the Active Directory tab and select the radio button for Yes to sync the AD users, groups, and to flatten the nested user groups into the parent group.
3. Add the Domain settings.
4. The AD sync schedule can be managed in the Timer tab in the Data Uploader or the sync can be manually triggered to run irrespective of the schedule.
5. The initial sync will bring all your Active Directory users and groups in CAM and update the configured external systems in the configured default database or repository. On subsequent data sync, the information is updated based on the updates in your Active Directory. For e.g. if you add a new user, the new user information is updated to the external system or if you delete a user from the AD, the user information is also disabled in the external systems i.e. any changes to your AD will also be updated in the configured external systems. The frequency of the updates is determined by the Timer tab settings.
6. The following AD properties are mapped (from CSV file) on data sync:

   AD Properties	Mapped As	Config file Properties
   First Name	Given Name	userEntry.FirstName
   Name	Name	userEntry.Name
   LastName	lastname	userEntry.LastName
   SamAccountName	samaccountname	userEntry.SamAccountName
   description	displayname	userEntry.DisplayName
   Description	description	userEntry.Description
   Email	mail	userEntry.Email
   Nested	nested	userEntry.Nested
   NestedGroups	nestedgroups	userEntry.NestedGroups
   ObjectGUID	Object GUID	userEntry.ObjectGUID
   Disabled	disabled	userEntry.Disabled
   Groups	memberOf	userEntry.Groups
   Telephone	telephoneNumber	UserEntry.Telephone
   Office	physicalDeliveryOfficeName	userEntry.Office
   StreetAddress	streetAddress	userEntry.StreetAddress
   City	city	userEntry.City
   PostalCode	postalCode	userEntry.PostalCode
   Country	country	userEntry.Country
   JobTitle	title	userEntry.JobTitle
   Department	department	userEntry.Depatment
   Company	company	userEntry.company
   DefaultDatabase	database
   DefaultCabinet	cabinet
   DefaultRole	role
   SyncNestedUsermembership	Should nested users be synced?	SyncNestedUsermembership

Filtering on the data brought in from Active Directory can be done on the configuration file in Data Uploader. This is so specific users or groups can be included or excluded from the sync. Multiple values are supported, but users cannot add multiple in one configuration.

• Wild cards are not supported, only standard operations like =,<,>,<=,>=, <> and !=.
• If the AD system has users from iManage, the users can be virtual, external or an AD user. These AD properties are supported, as well as domain and operating system.

The configuration for Active Directory lives in the Data.config file in the Prosperoware. Data Uploader folder (C:\Program Files (x86)\Prosperoware\Prosperoware Data Uploader\Data.config).

To set the config:

A sample configuration is below for users and groups:

<configuration>
                <maps>
                <map>
                <criteria>

<ad-property name="isHuman" value="TRUE"/>
                <ad-property name="name" value="1" operator="%%"/>
                </criteria>
                <values>
                <dms-property name="Database"  value="database_name"/>
                <dms-property name="Role"  value="Role_name"/>
                <include name="user" value="true"/>
                <include name="group" value="true"/>
                </values>
                </map>
                <map>
                </configuration>

The out of the box Data.config file for AD can be downloaded here as well.

A list of Active Directory ad-properties and attributes can be found on the Microsoft documentation here.

Data Uploader comes with the ability to sync CAM with an on premise local SQL database. Users may need the CAM data for third party databases like SPM. Additionally this locally stores the uploaded workspace jobs before uploading it to CAM. This enables an efficient Delta upload by allowing Data Uploader to compare the local data for changes and only upload those workspace jobs that really needs to be updated.

Follow the below steps to set this feature up:

To setup On Premise Sync:

Open Data Uploader.

Go to the On Premise Sync tab.

Enter the SQL server name. If it is on the localhost, you can put (local). Enter the Database Name.

On the Trusted Login option, select Yes or no. Yes is if the user will use Windows authentication to connect to SQL.

No lets the user set the UserId and Password.

Click Connect and Create Schema.

This creates the needed schema on the database for the CAM Data Uploader sync(s).

After the schema is created, set up the sync properties and schedule.

What do you want to sync?

• Users
• Groups and Group Membership
• Matter(Workspace)
• Metadata

How often do you want to sync?

• Minutes
• Hourly
• Daily
• Weekly
• Never- Means one time manually.

Schema of the On Premise Sync:

• CAMSync.GroupMembers: Contains the members of each group.
• CAMSync.Job- Contains the jobs that run.
• CAMSync.JobDetails- Details about which sync job is running. For example, is the job running a users job, matters job etc.
• CAMSync.Matter- The matters are synched here. Note these are just references. See Matter System table.
• CAMSync.MatterSystem- The details of the matters reside here. All system's matters live here (imanage, netdocuments, office365, external systems).
• CAMSync.Metadata-All of the metadata is synched here.
• CAMSync.UserGroups- The users and their groups are synched here.

The ad-property imported can be mapped to another property, so that the data comes through as desired.

To set the property mapping:

• Change the config for the AD sync to add the csvproperty.

For example, let's change the email to be the iManage userid and not the userid from Active Directory

The config should be setup like this:

<configuration>
                <fields>
                   <field adproperty="mail" csvProperty="UserId"/>
                </fields>
		 <groupfields>
		    <field adproperty="description" csvProperty="DisplayName"/>
		    <field adproperty="name" csvProperty="groupName"/>
		 </groupfields>
 </configuration>

• Within the adproperty field line, add csvProperty="Property" for it to use that csv property.
• Save the config.
• Run Data Uploader.

The email is set as the iManage userid, if you Generate CSV from Data Uploader, or run the Data Uploader Sync.

While exporting the document to network path with the option include CSV file, it will export the metadatas selected in the query.

The CSV values are generated according to the config file related to the external system.

Configuration path: C:\Program Files (x86)\Prosperoware\CAM\Data Uploader\service\config

Configure the fields according to the information below:

• name: The Name of NetDocuments\iManage Attribute
• description: The Name of the Column that will be shown on the CSV.
• attrId: The Attribute id in the NetDocuments\iManage.
• enabled: True or False(if true that column should be included in the csv and vice-versa.)
• prosp-parent: Specify the Parent of the metadata when having parent-child metadatas.

Sample files are attached:

iManage: IManageDocProperties.config

NetDocs: NetDocsProperties.config

Recommendations:

Firms are strongly recommended to move to using an on premise SQL database instead of SQLLite if one or more of the following is true:

• The firm has more than 5,000 active matters
• The firm is a medium to large sized company

This will allow Delta syncs to perform more efficiently as only the necessary workspaces and metadata will be updated.

SQL Requirements:

If the firm is using an on-premise SQL database, these are the requirements necessary:

• SQL Server 2012 Standard Edition or above
• A dedicated or Shared SQL server
• 8+GB of free RAM
• 4+CPU (cores/threads)
• 10+GB free hard disk space*
• *Solid state drive(s) are preferred for the SQL file storage