CrowdOp: Query Optimization For Declarative Crowd sourcing Systems

ABSTRACT

We study the query optimization problem in declarative crowdsourcing systems. Declarative crowd sourcing is designed to hide the complexities and relieve the user the burden of dealing with the crowd. The user is only required to submit an SQL-like query and the system takes the responsibility of compiling the query, generating the execution plan and evaluating in the crowd sourcing marketplace. A given query can have many alternative execution plans and the difference in crowd sourcing cost between the best and the worst plans may be several orders of magnitude. Therefore, as in relational database systems, query optimization is important to crowdsourcing systems that provide declarative query interfaces. In this paper, we propose CROWDOP, a cost-based query optimization approach for declarative crowdsourcing systems. CROWDOP considers both cost and latency in the query optimization objectives and generates query plans that provide a good balance between the cost and latency. We develop efficient algorithms in the CROWDOP for optimizing three types of queries: selection queries, join queries and complex selection-join queries. We validate our approach via extensive experiments by simulation as well as with the real crowd on Amazon Mechanical Turk.

AIM

The main aim of this paper is CROWDOP considers both cost and latency in the query optimization objectives and generates query plans that provide a good balance between the cost and latency

SCOPE

The scope of this paper is develop efficient algorithms in the CROWDOP for optimizing three types of queries: selection queries,    join queries and complex selection-join queries.

EXISTING SYSTEM

Recent crowdsourcing systems, such as CrowdDB, Qurk and Deco, provide an SQL-like query language as a declarative interface to the crowd. An SQL like declarative interface is designed to encapsulate the complexities of dealing with the crowd and provide the crowdsourcing system an interface that is familiar to most database users. Consequently, for a given query, a declarative system must first compile the query, generate the execution plan, post the human intelligence tasks (HITs) to the crowd according to the plan, collect the answers, handle errors and resolve the inconsistencies in the results. While declarative querying improves the usability of the system, it requires the system to have the capability to optimize and provide a “near optimal” query execution plan for each query. Since a declarative crowdsourcing query can be evaluated in many ways, the choice of execution plan has a significant impact on overall performance, which includes the number of questions being asked, the types/difficulties of the questions and the monetary cost incurred.

DISADVANTAGES

·      Our optimization objectives that consider both monetary cost and latency.

·      To efficiently select the best query plan with respect to the defined optimization objectives

 

PROPOSED SYSTEM

In this paper, propose CROWDOP, a cost-based query optimization approach for declarative crowdsourcing systems. CROWDOP considers both cost and latency in the query optimization objectives and generates query plans that provide a good balance between the cost and latency. We develop efficient algorithms in the CROWDOP for optimizing three types of queries: selection queries, join queries and complex selection-join queries. We validate our approach via extensive experiments by simulation as well as with the real crowd on Amazon Mechanical Turk.

ADVANTAGES    

1. The effective optimization algorithms for select, join and complex queries.
2. The  effectiveness of our query optimizer and validates our cost model and latency model.

 System Architecture

 

 

SYSTEM CONFIGURATION

HARDWARE REQUIREMENTS:-

·                Processor          -   Pentium –III

·                Speed                -    1.1 Ghz

·                RAM                 -    256 MB(min)

·                Hard Disk         -   20 GB

·                Floppy Drive    -    1.44 MB

·                Key Board                 -    Standard Windows Keyboard

·                Mouse               -    Two or Three Button Mouse

·                Monitor             -    SVGA

SOFTWARE REQUIREMENTS:-

·                Operating System      : Windows  7                                     

·                Front End                  : ASP.NET and C#

·                Database                   : MSSQL

·                Tool                           :Visual Studio

References                              

Meihui Zhang,Kok, S.,Meiyu Lu, Crowd Op: Query Optimization for Declarative Crowd sourcing Systems” IEEE Transactions on Knowledge and Data Engineering, Volume 27   Issue 8  March 2015.

A Stochastic Approach To Analysis Of Energy-Aware Dvs-Enabled Cloud Datacenters

ABSTRACT

With the increasing call for green cloud, reducing energy consumption has been an important requirement for cloud resource providers not only to reduce operating costs, but also to improve system reliability. Dynamic voltage scaling (DVS) has been a key technique in exploiting the hardware characteristics of cloud datacenters to save energy by lowering the supply voltage and operating frequency. This paper presents a novel stochastic framework for energy efficiency and performance analysis of DVS-enabled cloud. This framework uses virtual machine request arrival rate, failure rate, repair rate, and service rate of datacenter servers as model inputs. Based on a queuing network- based analysis, this paper gives analytic solutions of three metrics. The proposed framework can be used to help the design and optimization of energy-aware high performance cloud systems.

AIM

The main aim of this paper is presents a novel stochastic framework for energy efficiency and performance analysis of DVS-enabled cloud. 

SCOPE

The scope of this paper is a novel stochastic framework can be used to help the design and optimization of energy-aware high performance cloud systems.

EXISTING SYSTEM

To manage the applications in a cloud datacenter in an energy-efficient way becomes an urgent problem. DVS technologies give rise to a flexible solution to the above question. DVS tries to address the trade-off between performance and energy efficiency by taking into account two important characteristics of today’s computational systems: 1) the energy needed at the peak computing rate is much higher than the average one and 2) most today’s processors are based on CMOS logic. They suggest that high performance is needed only for a small fraction of the time in general, while for the rest of the time, a low-performance low-power pattern suffices. We can achieve the low performance by simply lowering the operating frequency of processors since the full speed mode is less energy-efficient. DVS goes beyond this and scales the operating voltage of processors along with the frequency. This is supported by today’s CMOS technology used by mainstream unicore/multicore processors.

DISADVANTAGES
·      Higher energy consumption

·      Reduce  operating costs

PROPOSED SYSTEM

In this paper proposes a queuing-network based framework for energy efficiency and performance evaluation of cloud datacenters with DVS capability. We consider expected VM completion time, VM loss rate, and energy consumption rate as key metrics of performance and energy efficiency. We employ a continuous-time Markov model to obtain analytical solutions of these metrics. To validate the effectiveness of their proposed model, we also conduct a case study on a sample datacenter with low-energy machines built on Intel X Scale PXA270 processors.

 

ADVANTAGES

DVS is used to quantify the effects of variations in workload, processor failure and recovery rates, the number of machines.

 DVS strategies and system capacity on performance and energy efficiency via the probabilistic analysis of continuous Markov chains.

System Architecture

SYSTEM CONFIGURATION

HARDWARE REQUIREMENTS:-

·                Processor          -   Pentium –III

·                Speed                -    1.1 Ghz

·                RAM                 -    256 MB(min)

·                Hard Disk         -   20 GB

·                Floppy Drive    -    1.44 MB

·                Key Board                 -    Standard Windows Keyboard

·                Mouse               -    Two or Three Button Mouse

·                Monitor             -    SVGA

SOFTWARE REQUIREMENTS:-

·                Operating System      : Windows  7                                     

·                Front End                  : ASP.NET and C#

·                Database                   : MSSQL

·                Tool                           :Visual Studio

References

YunNi Xia,MengChu Zhou,Xin Luo, ShanChen Pang “A Stochastic Approach to Analysis of Energy-Aware DVS-Enabled Cloud Datacenters” IEEE Transactions on Systems, Man, and Cybernetics: Systems Volume 45   Issue 1  August 2014.