LinkBack URL
About LinkBacks
Test Message Size Scalability in Symphony DE
1. Objective
The application I am developing will require passing large sized messages from the client to the service parts. Before I proceed, I want to see how Symphony DE and my system handles them.
2. Using Symping
With the Symping test tool, you can configure the size of Input, Output, as well as Common Data (also available from Platform Management Console):
The maximum message size which can be specified is 300000000 (268 MB). Result:Code:symping [-f configuration_file] [-u user_name] [-x password] [-a application_name] [-s session_type ] [-l [summary_file] ] [-T taskoverhead] [-i input_msg_size_in_bytes] [-o output_message_size_in_bytes ] [-r task_run_time_in_milliseconds] [-c common_data_size_in_bytes]
3. Using CodeCode:lechen@ib08b05-1169: symping -i 300000000 -o 300000000 -m 1 -d ------------------------------------------------------------ Symphony Ping Utility. Use this utility to test and verify that components are working and responsive. Vary the workload to try out different Symphony workload conditions. This client/service was developed using Symphony Developer Edition APIs. Configuration Value Input message size 300000000 bytes Output message size 300000000 bytes Common data size 0 bytes Client type async Number of test runs 1 Number of tasks per run 1 Task processing time 50 miliseconds Consume CPU cycles No Session type RecoverableAllHistoricalData ------------------------------------------------------------ Client connecting to Symphony... Connected to Session Director. Connection information: User: Admin Application: symping4.0 ------------------------------------------------------------ Connected to Session Manager. Creating 1 session... Session created. ID: 710 ------------------------------------------------------------ Test Run 1. Test for: Complete Workload Cycle Sending 1 tasks and retrieving replies... Number of tasks completed: 1 Connection to compute hosts OK. SIs and SIMs OK. Test Run 1 done. Test Roundtrip: 50.922 seconds ------------------------------------------------------------ Workload Summary: First task input sent: 2008-05-09 00:32:34.486132 Last task output received: 2008-05-09 00:33:23.871492 Total Task Roundtrip: 49.383 seconds [Last task output received - First task input sent] Number of compute hosts used: 1 Number of service instances used: 1 ------------------------------------------------------------ Task Roundtrip Summary: Shortest 49236.758 ms (task 1 on host ib08b05 ) Longest 49236.758 ms (task 1 on host ib08b05 ) Average 49236.758 ms ------------------------------------------------------------ Host Summary: Compute Host SI-PID Number of Tasks ib08b05 27221 1 ------------------------------------------------------------ Configuration Summary: Application: symping4.0 Session ID: 710
If you wish to test with even larger messages, or want to test the application you are developing, simply insert the following code. A large size string message (of "a"s) is generated, in place of the default "Hello Grid !!" input string. The large string will then be returned from the Service back to the Client (I used samples/CPP/SampleApp/SyncClient for demonstration):
Result, successfully sent and retrieved messages of 476 MB:Code:// Write large string to memory char* largeString; int msgSize = 500000000; largeString = (char*)malloc((msgSize+1)*sizeof(char)); memset(largeString, 'a', msgSize); largeString[msgSize] = 0; int tasksToSend = 1; for (int taskCount = 0; taskCount < tasksToSend; taskCount++) { // Create a message // char hello[] = "Hello Grid !!"; MyMessage inMsg(taskCount, true, largeString); // Create task attributes TaskSubmissionAttributes attrTask; attrTask.setTaskInput(&inMsg); // send it TaskInputHandlePtr input = sesPtr->sendTaskInput(attrTask); // Free memory largeString = NULL; inMsg.setString(""); // Retrieve and print task ID cout << "task submitted with ID : " << input->getId() << endl; } ...... // Bypass stdout to avoid overflow. Perform check sum if required. // cout << outMsg.getString() << endl;
View task details to confirm:Code:lechen@ib08b05-1210: Output/SyncClient connection ID=bb6fb596-ffff-ffff-c001-00112529883c-35445680-8886 Session ID:1301 task submitted with ID : 1 Task Succeeded [1] Integer Value : 0 All Done !!
4. TipsCode:lechen@ib08b05-1211: soamview task SampleAppCPP:1301:1 -l Task ID: 1 Task tag: Task status: done Application: SampleAppCPP Session ID: 1301 Submitted: May 08 21:33:06.926 Started: May 08 21:33:12.496 Ended: May 08 21:36:28.420 Input message size: 500000024 bytes (476.84 MB) Output message size: 500000084 bytes (476.84 MB) Retried: 0 Compute host: ib08b05 Service instance PID: 29616 Last failure reason: - Control reason:
Even though Symphony DE can handle large size input and output messages, developers should take advantage of data parallelism to improve the application's performance and to maximize resource utilization.
Partitioned data can be processed in parallel on different hosts and will finish running sooner than the aggregated data processed in one shot or sequentially on one host.
Thus it's desirable to have input data that can be easily broken up into smaller chunks. Each of these chunks can be processed by a Symphony task. Symphony task is a small unit of work, typically implemented by a function or class-method. Each task has its own set of inputs and outputs. See Ajith's article for details: Symphony DE Application Performance.
5. Related Articles
Comments welcomed.
Reply With Quote