Quantcast
Channel: Business Intelligence y Big Data: ¡Aprende Gratis sobre Analytics!
Viewing all 866 articles
Browse latest View live

STCard, a free license way to create powerful Scorecards

$
0
0


The improvements in this version of STCard, an open source based solution, are focused on user interface for panel and dashboard and also some enhancement in performance and close some old bugs:

- Import with ETL
- New KPIs always in red bug
- Tooltips and characters solved
- Export to PDF
- Modify colors of new scorecard
- Some other minus bugs...

It works with Pentaho and embeded in web applications

You can manage your organization with a powerful KPIs control with Balance Scorecard using STCard

You can see it in action in this Pentaho Demo Online and as a part of LinceBI suite

STCard doesn´t requiere anual license, you can manage unlimited users and it´s open source based. 

STCard includes professional services (training, support and maintenance, docs and bug resolution - so, you have high enterprise level guaranteed -)

Interested? contact Stratebi or LinceBI



See a Video Demo:




About main functionalities:

STCard works on top of Pentaho, is the best tool for managing your KPIs (Key Performance Indicators), targets an keep track of your Balance Scorecard strategy









Fully integrated with Pentaho CE, you can leverage all the power of this Open Source BI Suite



STCard is an open source tool developed by StrateBI for the creation, management and analysis of Scorecards.
A Scorecard is a global management system within an organization that allows you to have a view of it based on a number of perspectives. All these as a whole define the vision and strategy of the organization.
To define a Scorecard you have to define a clear strategy:
  • Strategic Objectives for the units of the organization.
  • Indicators (KPI’s) that mark the fulfillment of the strategic objectives.
The main features of STCard are:
  • Flexibility: A Scorecard is always referred to an organization as a whole, but with STCard we can create a scorecard for a specific area of the organization. For example:Treasury Financial Area, Consolidation, Suppliers, etc. On the other hand, the concept of flexibility is applicable to the creation of a scorecard in terms of the number of strategic perspectives and objectives. As many as you like. The philosophy of Kaplan and Norton is not limited to 4 perspectives: customer, financial, internal business procedures and learning and growth. You can create as you need
  • Flexibility does not break with the original philosophy. A scorecard in STCAD consists of a weighted hierarchical structure of 3 levels:
    • Perspective: from what point of view we will see our system. For example, financial, quality, customers, IT, etc.
    • Strategic Objective: what is our goal. For example, increase profitability, customer loyalty, incentive and motivation HR, etc.
    • Indicator (KPI): the measure or metric. Indicators can be quantitative or qualitative (confirmation / domain values), and these always have a real value and a target value.
For the launch of the ScoreCard we can consider three scenarios:
  • This scenario has a rapid implementation, and only requires the definition of a load processes to obtain the information of the indicators of the organization and adapt it to STCard.
  • The organization lacks a system / repository of indicators.
    This variant requires more consulting work, because in the organization, first, a pure BI project must be carried out to obtain those indicators to be dealt with later in STCARD.
    For example: data sources; ETL processes; System / repository of indicators; Load processes in STCard.
  • Immediate start-up:
    It is the fastest alternative, only requires installation / configuration and training. Data management is done through Excel templates. No additional consulting work required.
    Users set values through Excel templates, where data is filled. These values are loaded into STCARD and after this, it is the users who interact with STCARD.

These are the main features of STCard:




Tips y Tecnicas de optimización de Vertica con Talend

$
0
0

Os traemos unos cuantos trucos y recomendaciones sobre dos de nuestras herramientas favoritas: Vertica y Talend

Configuring Talend for Use with Vertica
To configure Talend for use with Vertica, you must understand:

Using Talend Components for Vertica

Talend provides a number of Vertica-specific components that you can use to manage your ETL jobs. Understanding the ETL flow and loading guidelines can help you determine which components to use, and how best to use them.

Understanding ETL Flow and Loading Guidelines

Understanding the basic ETL flows helps you decide which Vertica component and options to use. This section addresses the following load types:


Type of Load Use this COPY Option Results
Small Bulk Load COPY (<100mb font="">100mb>AUTO
  • Writes to WOS.
  • Spills to ROS when WOS overflows.
Large Bulk Load COPYDIRECT
  • Writes to ROS.
  • Each commit becomes a new ROS container.
Incremental Load COPYTRICKLE
  • Writes to WOS.
  • Errors when WOS overflows.
Follow these guidelines when loading data into Vertica:
  • Use COPY to load large amounts of data. Using COPY avoids fragmenting the WOS and doesn't require the overhead that using INSERT does.
  • If your row count is small (fewer than 1000 rows), use INSERT.
  • Load multiple streams on different nodes.
If you are using the INSERT INTO...SELECT...FROM syntax in ETL jobs with large volumes, be sure to use the following syntax.


=> INSERT /+*direct*/ INTO table SELECT…

Commonly Used Vertica Specific Talend Components

The following Talend 6.4.1 components are specifically for use with Vertica:
Component Description
tVerticaInputExtracts data from Vertica.
tVerticaBulkExecLoads from a file.
tVerticaOutputBulkWrites to a file.
tVerticaOutputBulkExecWrites to a file and loads the data.
tVerticaOutputInserts or updates rows into a Vertica table.
tVerticaRowExecutes the SQL query stated against the Vertica database.

tVerticaInput

The tVerticaInput component allows you extract data from Vertica. Give special attention to the DATE and VARCHAR fields, as follows: 
  • DATE fields—All DATE fields must be in MM-DD-YYYY format. Check your DATE fields and correct the format wherever necessary, as shown in the following dialog box.
image003.png
  • VARCHAR fields—Talend examines a sampling of the data when determining the size of the VARCHAR field. If the table contains large VARCHAR values, consider increasing the default column size.
Note

 You can also use the tMap component to transform or remap the data type.

tVerticaOutput

tVerticaOutput provides options that you can change, depending on the size and type of load. Specifically, you should change the Action on Data and Action on Table options, as described in the following sections.
Basic Settings for Trickle or Incremental Loads (Writing to WOS)
Set the Actionon Table and Action on Data options according to your use case. For trickle or incremental loads, set Action on Data to INSERT.

image004.png
Advanced Settings for Trickle or Incremental Loads (Writing to WOS)
When writing to the WOS, use the default values for Advanced Settings as follows.

image005.png

To enable fields for INSERT, UPDATE, and DELETE, select the Use field options checkbox. The default settings in the previous graphic generate the following SQL statements:


2018-05-15 05:24:36.797 Init Session:7f17b3fff700 [Session]  [PQuery]
TX:0(v_target_l1_node0001-40400:0x1091f) INSERT INTO public.VarChar4k_Table
(DataTypeSet,ValueDesc,VarChar4k_Column) VALUES (?,?,?)

Note

These default settings do not generate a COPY statement.
Basic Settings for Large, Bulk Loads (Writing to ROS)
For large, bulk loads, use the DIRECT syntax to write to ROS. To do so, you must set the Action on data option to COPY. 

image006.png
Advanced Settings for Large, Bulk Loads (Writing to ROS)
When using COPY, use Advanced Settings to specify filenames and locations for exception files.

image007.png

The settings in the preceding graphic generate the following SQL statements:


2018-05-15 05:19:44.584 Init Session:7f17fd7ca700-a0000000044bf8 [Txn] 
Rollback Txn: a0000000044bf8 'COPY public.VarChar4k_Table
(DataTypeSet,ValueDesc,VarChar4k_Column) FROM local STDIN UNCOMPRESSED WITH DELIMITER ';'
RECORD TERMINATOR '' ENCLOSED BY '|' NULL AS '' EXCEPTIONS 'C:/data/except_file.txt'
REJECTED DATA 'C:/data/except_records.txt' DIRECT NO COMMIT'

tVerticaOutputBulkExec

The tVerticaOutputBulkExec component writes to a file and then copies the data using standard input (STDIN).
Basic Settings
image008.png
Advanced Settings
By default, tVerticaOutputBulkExec writes to the ROS.

image009.png

The settings in the preceding graphic result in the following SQL statements:


2018-05-15 05:20:44.526 Init Session: 7f17b37fe700-a0000000044c10
[Txn]
Starting Commit: Txn: a0000000044c2d 'COPY
datatype_ss.VarChar_Table FROM local STDIN DELIMITER ';' NULL 'null'
DIRECT returnrejected'

tVerticaRow

The tVerticaRow component allows you to specify any valid Vertica SQL statements, including COPY statements.  Use tVerticaRow to load data into Vertica flex tables, or for other scenarios that require Vertica structures that are not supported with custom Talend components. 
Example: High-Performance COPY
In the following example, the source file is on the Vertica server and uses the high-performance Vertica COPY (not COPY FROM LOCAL). Whenever the source files are on the Vertica cluster, load data using high-performance COPY.
"COPY talend.store_target FROM '/home/dbadmin/store.csv' DELIMITER ';'
NULL '' DIRECT;"
Example: Loading into Flex Tables
This example shows how to run the Vertica flex example included in the Vertica package directory.
To create the flex table:
=> CREATE FLEX TABLE mountains();
To load data to the flex table:
=> COPY mountains FROM 
'/opt/vertica/packages/flextable/examples/mountains.json'
   PARSER FJSONPARSER();
To create the view of the flex table:
=> SELECT COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW('mountains');
Example:  Performing Vertica-to-Vertica COPY
This example shows a Vertica-to-Vertica COPY:


“CONNECT TO VERTICA vmart USER dbadmin PASSWORD '' ON 
'VertTest01',5433;COPY customer_dimension FROM  VERTICA
vmart.customer_dimension DIRECT;”

You can also use tVerticaRow to copy from Vertica to Vertica when no transformations are required. To do so, you need to perform additional steps to define the export addresses.
Connecting to a public network requires some configuration. For information about using this statement to copy data across a public network, see Using Public and Private IP Networks in the Vertica documentation.

Using Generic Talend ELT Components with Vertica

The ELT family of components group together database connectors and processing components for ELT mode, where the target DBMS becomes the transformation engine.  When possible, SQL statements are combined and processed in a single query on the database. The following illustration shows how Talend supports ELT with Vertica. This example uses the generic components for the entire job. 

image011.png
When this job runs, Talend generates the following SQL statements:


=> INSERT  INTO store.regional_sales (store_name,store_city,store_region,sales_quantity,
sales_dollar_amount)

(SELECT Store.store_dimension.store_name , Store.store_dimension.store_city ,
Store.store_dimension.store_region , sum(store.store_sales_fact.sales_quantity),
sum(store.store_sales_fact.sales_dollar_amount) FROM Store.store_dimension INNER JOIN
store.store_sales_fact ON

( store.store_sales_fact.store_key = Store.store_dimension.store_key )

group by Store.store_dimension.store_name,
Store.store_dimension.store_city,Store.store_dimension.store_region )

Other Components for Bulk Loading

The tVerticaOutputBulk and tVerticaBulkExec components are generally used together in a two-step process. First, an output file is generated. In the second step, the output file is used in the INSERT operation that loads data into a database.
You can also choose to do both steps at once using the tVerticaOutputBulkExec component.  However, using tVerticaOutputBulk and tVerticaBulkExec allows the data to be transformed before it is loaded in the database.

Using the Talend SQL Builder

When using the SQL builder, be sure to include the schema name in the query:
=> SELECT * FROM store.store_dimension;

Enabling Parallelization in Talend

You can enable parallelized data flows. To do so, partition an input data flow of a Talend subjob into parallel processes and execute these processes simultaneously.    

Enabling Parallelization in the Talend Enterprise Edition

If you are using the Talend Enterprise Edition, you can enable or disable the parallelization with a single click. Talend Studio then automates the implementation across a given job.  For more information about this feature, click the following link:

Enabling Parallelization in the Talend Community Edition

If you are using the Talend Community Edition, add a WHERE clause to the original query to chunk the data.  This example results in four chunks.
original_sql + " and hash(" + primaryKey + ") % " + noOfThreads + " = " + i
Example:


=> SELECT if.* FROM inventory_fact if, warehouse_dimension wd WHERE
   if.warehouse_key=wd.warehouse_key

The preceding query chunks the SELECT operation into the following four queries:


=> SELECT if.* FROM inventory_fact if, warehouse_dimension wd WHERE
  if.warehouse_key=wd.warehouse_key AND HASH(product_key, date_key) % 4=1;
=> SELECT if.* FROM inventory_fact if, warehouse_dimension wd WHERE
  if.warehouse_key=wd.warehouse_key AND HASH(product_key, date_key) % 4=2;
=> SELECT if.* FROM inventory_fact if, warehouse_dimension wd WHERE
  if.warehouse_key=wd.warehouse_key AND HASH(product_key, date_key) % 4=3;
=> SELECT if.* FROM inventory_fact if, warehouse_dimension wd WHERE
  if.warehouse_key=wd.warehouse_key AND HASH(product_key, date_key) % 4=4;
Choose keys that have equal distribution. For example, the two keys chosen in the preceding example provide the following counts:

Key : 235164 Value : product_key , Key : 50148 Value : date_key
count, chunk
7501441, 1
7500008, 2
7497028, 0
7501523, 3

Cual es el orden correcto para hacer un proyecto de Machine Learning?

Formacion Data Engineer: Machine Learning con Spark

$
0
0

Uno de los capítulos más interesante del curso 'Curso de Data Engineers'de nuestros compañeros de Stratebi es el 11. Machine Learning con Spark

Este curso incluye ejercicios prácticos sobre nuestro Cluster. Os dejamos unas pinceladas del contenido de este capítulo:








El martes 26 de Noviembre, no te pierdas AlicanteData

$
0
0

Toda una mañana, con un gran panel de ponentes sobre Datos y Analytics en Alicante, el martes 26 de Noviembre. No te lo pierdas

Tras los eventos en Madrid, Barcelona y Sevilla, ahora en Alicante!!


Temario completo. Una mañana muy bien aprovechada!!

Un gran panel de Ponentes

DESCUBRE LAS TECNOLOGÍAS BUSINESS INTELLIGENCE MÁS COMPLETAS Y QUE ESTÁN LIDERANDO LOS PROYECTOS HOY EN DÍA. ASÍ COMO SUS CARACTERISTICAS, DE LA MANO DE LOS GRANDES ESPECIALISTAS DEL SECTOR.


APRENDE SOBRE TALEND, Big Data, AI , Machine learning, VERTICA Y POWERBI (LA COMBINACION MÁS COMPLETA Y POTENTE) PARA BI, BIG DATA Y ANALYTICS.



Videotutorial: Usando R para Machine Learning con PowerBI

$
0
0

Si te gusta el Business Intelligence, Machine Learning y conoces algo de R, no te puedes perder este Videotutorial:




Temario del Videotutorial:

1º Posibilidades de R dentro de Power BI 
2º Instalar R y RStudio 
3º Matriz de Correlación para el sector Retail 
4º Árbol Decisión Titanic 
5º Clustering K-Means Estados Unidos 
6º Clustering DBSCAN Estados Unidos para detectar Outliers 
7º Forecasting Exponencial Smoothing Births Nacimientos 
8º Descomposición de Series Temporales Nacimientos 
9º Power BI R Visual con ggplot2

Para Saber más de PowerBI:


Big Data para PowerBI



Power BI es un conjunto de herramientas Business Intelligence (BI) desarrolladas por Microsoft. De reciente aparición, gracias a su simplicidad y potencia se ha hecho un un hueco entre las grandes del mercado como Tableau, Pentaho o Microstrategy.  Al igual que estas últimas, implementa la filosofía de Autoservicio para el usuario final (Self Service BI) llevada al extremo de la sencillez, pero con un gran número de características...

Working together PowerBI with the best open source solutions



Here you can see a nice sample combining PowerBI with open source based Business Intelligence solutions, like LinceBI, in order to provide the most complete BI solution with an affordable cost - Predefined Dashboards - Adhoc Reporting - OLAP Analysis - Adhoc Dashboarding - Scorecards More info: - PowerBI functionalities - PowerBI traini...


Comparativa de Costes Tableau vs PowerBI



  Os dejamos un documento listo para descargar, con una comparativa muy completa de costes entre Tableau y PowerBI (hay que decir que el informe ha sido encargado por Tableau, por lo que puede tener cierto sesgo).  Por ejemplo, en cuanto al esfuerzo de este tipo de proyectos, si tenemos en cuenta que ambas son herramientas de Data Discovery (usuario final), no se tiene suficientemente en cuenta la parte más importante, el modelado,...

Viewing all 866 articles
Browse latest View live