Data Engineering on Microsoft Azure Test 14

Required: Utilize a solution which will deploy multiple HDInsight clusters with minimal effort.
Which of the following should recommend to the IT team to implement?

Select 1 option(s):

Which of the following are valid deployment methods? (Select two)

Select 2 option(s):

True or False: It is possible to use multiple languages in one notebook.

Select 1 option(s):

When a table is created, by default the data structure has no indexes and is called a(n) [?].

Select 1 option(s):

Select 1 option(s):

You narrowed the choices down to the below list.
Which is the best choice?

Select 1 option(s):

True or False: Enabling analytical store is only available at the time of creating a container however it can be deactivated or reactivated at anytime thereafter.

Select 1 option(s):

• Firewall rules
• Virtual networks
• Private endpoints
Which of the following are benefits of using a managed workspace virtual network? (Select all that apply)

Select 4 option(s):

Select 1 option(s):

Which of the following are true about Spark pools in Azure Synapse Analytics? (Select all that apply)

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Security administrators can control data access by using [?] within Data Lake Storage. Built-in security groups include ReadOnlyUsers, WriteAccessUsers, and FullAccessUsers.

Select 1 option(s):

Azure Databricks is a fully-managed, cloud-based Big Data and Machine Learning platform, which empowers developers to accelerate AI and innovation by simplifying the process of building enterprise-grade production data applications. Built as a joint effort by Databricks and Microsoft, Azure Databricks provides data science and engineering teams with a single platform for Big Data processing and Machine Learning.
A Microsoft-managed Azure Databricks workspace virtual network (VNet) exists within the customer subscription. Information exchanged between this VNet and the Microsoft-managed Azure Databricks Control Plane VNet is sent over a secure TLS connection through ports (22 and 5557) that are enabled by Network Security Groups (NSGs) and protected with port IP filtering.
The Blob Storage account provides default file storage within the workspace (databricks file system (DBFS)). This resource and all other Microsoft-managed resources are completely locked from changes made by the customer.
True or False: You can write to the default DBFS file storage as needed, but you cannot change the Blob Storage account settings.

Select 1 option(s):

Select 1 option(s):

An Azure integration runtime is capable of which of the following? (Select all that apply)

Select 3 option(s):

Often, due to compliance, the caller has to identify themselves by giving them the last four digits of their credit card number that they may have an issue with. These data items cannot be fully exposed to the service representative in that call centre.
Which type of security would typically be best used in for this scenario?

Select 1 option(s):

Currently, you are in a meeting with the IT team and discussing a project to pull data from SQL Server and migrate it to Azure Blob storage.
Required:
• The process must orchestrate and manage the data lifecycle.
• The process must configure Azure Data Factory to connect to the on-premises SQL Server database.
Ray and the IT team have put together a list of actions they think need to be performed to meet the needs of the project, but they are not sure on the order to execute. Below is a list of the actions they are considering.
Proposed Actions:
a. Create an Azure Data Factory resource.
b. Configure a self-hosted integration runtime.
c. Create a virtual private network (VPN) connection from on-prem to MS Azure.
d. Create a database master key on SQL Server.
e. Backup the database and send it to Azure Blob storage.
f. Configure the on-prem SQL Server instance with an integration runtime.
As you are the Azure SME, Ray and the team look to you for direction on selecting the required items and putting them in the proper order. Which of the below contains the correct items in the correct sequence to meet the requirements?

Select 1 option(s):

A(n) [?] schema must be defined before query time.

Select 1 option(s):

Sharing data from one worker to another can be a costly operation.
Spark has optimized this operation by using a format called Tungsten which prevents the need for expensive serialization and de-serialization of objects in order to get data from one JVM to another.
The data that is “shuffled” is in a format known as UnsafeRow, or more commonly, the Tungsten Binary Format.
When we shuffle data, it creates what is known as a stage boundary which represents a process bottleneck which Spark will break this one job into two stages.
In Stage #1, Spark will create a pipeline of transformations in which the data is read into RAM.
For Stage #2, Spark will again create a pipeline of transformations in which the shuffle data is read into RAM.
From the developer’s perspective, we start with a read and conclude (in this case) with a write:
Step Transformation
1 Read
2 Select
3 Filter
4 GroupBy
5 Select
6 Filter
7 Write
However, Spark starts with the action (write(..) in this case).
What is the main benefit of working backward through your action’s lineage?

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Review the following architecture designs.
Design A:

Design B:

Design C:

Which design would be best suited for the need?

Select 1 option(s):

The new solution must be on a petabyte scale that doesn’t involve complex installations and configurations.
Which of the following would best suit the need?

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

PowerShell
$SubscriptionId = “add your subscription here”
Add-AzureRmAccount
Set-AzureRmContext -SubscriptionId $SubscriptionId
Register-AzureRmResourceProvider -ProviderNamespace Microsoft.DataFactory
$resourceGroupName = “cto_ignite”
$rglocation = “West US 2”
New-AzureRmDataFactoryV2 -ResourceGroupName $resourceGroupName -Name “ctoigniteADF” -Location $rglocation
What is this code template used to setup?

Select 1 option(s):

Many business application architectures separate transactional and analytical processing into separate systems with data stored and processed on separate infrastructures. [?] systems are optimized for dealing with discrete system or user requests immediately and responding as quickly as possible.

Select 1 option(s):

The company has turned to you for help with creating an architecture that can scale with the data needs that are required to create a predictive model and to handle the JSON data so that it’s integrated into the BI solution.
Review the following architecture designs.
Design A:

Design B:

Design C:

Which architecture would be best suited for the need?

Select 1 option(s):

Select 1 option(s):

Select 1 option(s):

Which of the following can be used to do this?

Select 1 option(s):

When you execute code, Spark SQL uses Catalyst’s general tree transformation framework in four phases, as shown below:
1. analyzing a logical plan to resolve references
2. logical plan optimization
3. physical planning
4. code generation to compile parts of the query to Java bytecode
In the physical planning phase, Catalyst may generate multiple plans and compare them based on [?].

Select 1 option(s):

These methods are:
• Mapping Data Flows
• Compute Resources
• SSIS Packages
Mapping Data Flows provides a number of different transformations types that enable you to modify data. They are broken down into the following categories:
• Schema modifier transformations
• Row modifier transformations
• Multiple inputs/outputs transformations
Which transformations type is best described by:
“A Sort transformation that orders the data.”

Select 1 option(s):

Which of the Azure Storage account options is best described by:
“Support all of the latest features for blobs, files, queues, and tables. Pricing has been designed to deliver the lowest per gigabyte prices.”

Select 1 option(s):

Select 1 option(s):

Bruce has come to you because his IT team needs advice on the use of Azure SQL Database to support a mission-critical application.
Required:
• The application must be highly available
• No performance loss during maintenance cycles
Which of the following applications should you recommend to Bruce and his team to adopt?

Select 3 option(s):

Select 6 option(s):

Which of the following is the best approach if singleton or smaller transaction batch loads must be added to an MPP data warehouse?

Select 1 option(s):

Select 1 option(s):

Which Azure Data Factory integration runtime should be used in a data copy activity?

Select 1 option(s):

[?] is a fully managed cloud service. Analysts, data scientists, developers, and others use [?] to discover, understand, and consume data sources. It features a crowdsourcing model of metadata and annotations. In this central location, an organization’s users contribute their knowledge to build a community of data sources that are owned by the organization.

Select 1 option(s):

• Provide undoubtedly the most well-understood model for holding data.
• The simplest structure of columns and tables makes them very easy to use initially, but the inflexible structure can cause some problems.
• We can communicate with relational databases using SQL.

Select 1 option(s):

• Data movement activities
• Data transformation activities
• Control activities
When using JSON notation, the activities section can have one or more activity defined within it.
They have the following top-level structure:
JSON
{
“name”: “Execution Activity Name”,
“description”: “description”,
“type”: ““,
“typeProperties”:
{
},
“linkedServiceName”: “MyLinkedService”,
“policy”:
{
},
“dependsOn”:
{
}
}
Which of the JSON properties are required for HDInsight? (Select all that apply)

Select 4 option(s):

Select 6 option(s):

Which command filters based on the end of a column value as required?

Select 1 option(s):

Select 1 option(s):

What will the result be of the pipeline?

Select 1 option(s):

Azure provides many ways to store your data. A storage account is a(n) [?] that groups a set of Azure Storage services together.

Select 1 option(s):

Roles such as business stakeholders, business analysts, and business intelligence developers are well known and valuable.
Which of the available roles is best described by:
“Works with artificial intelligence services such as Cognitive Services, Cognitive Search, and Bot Framework. Cognitive Services includes Computer Vision, Text Analytics, Bing Search, and Language Understanding (LUIS).
Rather than creating models, they apply the pre-built capabilities of Cognitive Services APIs. Part of their job is to embed these capabilities within a new or existing application or bot. They rely on the expertise of data engineers to store information that’s generated from artificial intelligence.”

Select 1 option(s):

True or False: For virtual machines (VMs), Azure lets you encrypt virtual hard disks (VHDs) by using Azure Disk Encryption. If someone gets access to the VHD image and downloads it, they can’t access the data on the VHD unless they have an Azure Storage account as well. If a bad actor restores the image within their own Azure environment, they will have access to the data on the image.

Select 1 option(s):

You can set permissions at a directory level or file level for the data stored within the data lake. This security is configurable through technologies such as Hive and Spark, or utilities such as Azure Storage Explorer. All data that is stored is encrypted at rest by using either Microsoft or customer-managed keys.
Data Lake Storage Gen2 supports which of the following to enhance security?

Select 2 option(s):

Azure Monitor provides base-level infrastructure metrics and logs for most Azure services. Azure diagnostic logs are emitted by a resource and provide rich, frequent data about the operation of that resource. Azure Data Factory (ADF) can write diagnostic logs in Azure Monitor.
Data Factory stores pipeline-run data for [?] days.

Select 1 option(s):

Which icon should you click on the home page of Azure Synapse Studio to begin the integration?

Select 1 option(s):

Within Azure Synapse SQL, [?] stores a copy of the result set on the control node so that queries do not need to pull data from the storage subsystem or compute nodes.

Select 1 option(s):

• A shared Hive-compatible metadata system enables you to define tables on files in the data lake such that it can be consumed by either Spark or Hive.
• Both SQL and Spark can directly explore, and analyze Parquet, CSV, TSV, and JSON files stored in the data lake.
• The enablement of fast scalable load and unload for data transferring between SQL and Spark databases.
The Azure Synapse Apache Spark to Synapse SQL connector is designed to efficiently transfer data between serverless Apache Spark pools and SQL pools in Azure Synapse.
Which of the following are valid use cases for Apache Spark and SQL integration within Synapse analytics? (Select all that apply)

Select 4 option(s):

Azure Synapse Analytics can work by acting as the one stop shop to meet all of your analytical needs in an integrated environment.
[?] leverages the capabilities of Azure Data Factory and is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using [?], you can create and schedule data-driven workflows that can ingest data from disparate data stores.

Select 1 option(s):

What does CI/CD focus on?

Select 1 option(s):

• Too many small or very big files – more time opening & closing files rather than reading contents (worse with streaming).
• Partitioning also known as “poor man’s indexing”- breaks down if you picked the wrong fields or when data has many dimensions, high cardinality columns.
• No caching – cloud storage throughput is low (cloud object storage is 20-50MB/s/core vs 300MB/s/core for local SSDs).
As a solution to the challenges with Data Lakes noted above, [?] is a file format that can help you build a data lake comprised of one or many tables in [?] format. [?] integrates tightly with Apache Spark, and uses an open format that is based on Parquet. Because it is an open-source format, [?] is also supported by other data platforms, including Azure Synapse Analytics.

Select 1 option(s):

Spark is a Distributed computing environment. The unit of distribution is a Spark Cluster. Every Cluster has a Driver and one or more executors. Work submitted to the Cluster … [?]

Select 1 option(s):

Which of the following are supported in the HDInsight solution? (Select all that apply)

Select 6 option(s):

Select 1 option(s):