Pig UDF Manual
Overview
Pig provides extensive support for user-defined functions (UDFs) as a way to specify custom processing. Functions can be a part of almost every operator in Pig. This document describes how to use existing functions as well as how to write your own functions.
Eval Functions
How to Use a Simple Eval Function
Eval is the most common type of function. It can be used in FOREACH statements as shown in this script:
-- myscript.pig REGISTER myudfs.jar; A = LOAD 'student_data' AS (name: chararray, age: int, gpa: float); B = FOREACH A GENERATE myudfs.UPPER(name); DUMP B;
The command below can be used to run the script. Note that all examples in this document run in local mode for simplicity but the examples can also run in Hadoop mode. For more information on how to run Pig, please see the PigTutorial.
java -cp pig.jar org.apache.pig.Main -x local myscript.pig
The first line of the script provides the location of the jar file that contains the UDF. (Note that there are no quotes around the jar file. Having quotes would result in a syntax error.) To locate the jar file, Pig first checks the classpath. If the jar file can't be found in the classpath, Pig assumes that the location is either an absolute path or a path relative to the location from which Pig was invoked. If the jar file can't be found, an error will be printed: java.io.IOException: Can't read jar file: myudfs.jar.
Multiple register commands can be used in the same script. If the same fully-qualified function is present in multiple jars, the first occurrence will be used consistently with Java semantics.
The name of the UDF has to be fully qualified with the package name or an error will be reported: java.io.IOException: Cannot instantiate:UPPER. Also, the function name is case sensitive (UPPER and upper are not the same). A UDF can take one or more parameters. The exact signature of the function should clear from its documentation.
The function provided in this example takes an ASCII string and produces its uppercase version. If you are familiar with column transformation functions in SQL, you will recognize that UPPER fits this concept. However, as we will see later in the document, eval functions in Pig go beyond column transformation functions and include aggregate and filter functions.
If you are just a user of UDFs, this is most of what you need to know about UDFs to use them in your code.
How to Write a Simple Eval Function
Let's now look at the implementation of the UPPER UDF.
package myudfs; import java.io.IOException; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; import org.apache.pig.impl.util.WrappedIOException; public class UPPER extends EvalFunc (String) { public String exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; try{ String str = (String)input.get(0); return str.toUpperCase(); }catch(Exception e){ throw WrappedIOException.wrap("Caught exception processing input row ", e); } } }
The first line indicates that the function is part of the myudfs package. The UDF class extends the EvalFunc class which is the base class for all eval functions. It is parameterized with the return type of the UDF which is a Java String in this case. We will look into the EvalFunc class in more detail later, but for now all we need to do is to implement the exec function. This function is invoked on every input tuple. The input into the function is a tuple with input parameters in the order they are passed to the function in the Pig script. In our example, it will contain a single string field corresponding to the student name.
The first thing to decide is what to do with invalid data. This depends on the format of the data. If the data is of type bytearray it means that it has not yet been converted to its proper type. In this case, if the format of the data does not match the expected type, a NULL value should be returned. If, on the other hand, the input data is of another type, this means that the conversion has already happened and the data should be in the correct format. This is the case with our example and that's why it throws an error (line 16.) Note that WrappedIOException is a helper class to convert the actual exception to an IOException.
Also, note that lines 10-11 check if the input data is null or empty and if so returns null.
The actual function implementation is on lines 13-14 and is self-explanatory.
Now that we have the function implemented, it needs to be compiled and included in a jar. You will need to build pig.jar to compile your UDF. You can use the following set of commands to checkout the code from SVN repository and create pig.jar:
svn co http://svn.apache.org/repos/asf/hadoop/pig/trunk cd trunk ant
You should see pig.jar in your current working directory. The set of commands below first compiles the function and then creates a jar file that contains it.
cd myudfs javac -cp pig.jar UPPER.java cd .. jar -cf myudfs.jar myudfs
You should now see myudfs.jar in your current working directory. You can use this jar with the script described in the previous section.
Aggregate Functions
Aggregate functions are another common type of eval function. Aggregate functions are usually applied to grouped data, as shown in this script:
-- myscript2.pig A = LOAD 'student_data' AS (name: chararray, age: int, gpa: float); B = GROUP A BY name; C = FOREACH B GENERATE group, COUNT(A); DUMP C;
The script above uses the COUNT function to count the number of students with the same name. There are a couple of things to note about this script. First, even though we are using a function, there is no register command. Second, the function is not qualified with the package name. The reason for both is that COUNT is a builtin function meaning that it comes with the Pig distribution. These are the only two differences between builtins and UDFs. Builtins are discussed in more detail later in this document.
An aggregate function is an eval function that takes a bag and returns a scalar value. One interesting and useful property of many aggregate functions is that they can be computed incrementally in a distributed fashion. We call these functions algebraic. COUNT is an example of an algebraic function because we can count the number of elements in a subset of the data and then sum the counts to produce a final output. In the Hadoop world, this means that the partial computations can be done by the map and combiner, and the final result can be computed by the reducer.
It is very important for performance to make sure that aggregate functions that are algebraic are implemented as such. Let's look at the implementation of the COUNT function to see what this means. (Error handling and some other code is omitted to save space. The full code can be accessed here.
public class COUNT extends EvalFunc (Long) implements Algebraic{ public Long exec(Tuple input) throws IOException {return count(input);} public String getInitial() {return Initial.class.getName();} public String getIntermed() {return Intermed.class.getName();} public String getFinal() {return Final.class.getName();} static public class Initial extends EvalFunc (Tuple) { public Tuple exec(Tuple input) throws IOException {return TupleFactory.getInstance().newTuple(count(input));} } static public class Intermed extends EvalFunc (Tuple) { public Tuple exec(Tuple input) throws IOException {return TupleFactory.getInstance().newTuple(sum(input));} } static public class Final extends EvalFunc (Long) { public Tuple exec(Tuple input) throws IOException {return sum(input);} } static protected Long count(Tuple input) throws ExecException { Object values = input.get(0); if (values instanceof DataBag) return ((DataBag)values).size(); else if (values instanceof Map) return new Long(((Map)values).size()); } static protected Long sum(Tuple input) throws ExecException, NumberFormatException { DataBag values = (DataBag)input.get(0); long sum = 0; for (Iterator (Tuple) it = values.iterator(); it.hasNext();) { Tuple t = it.next(); sum += (Long)t.get(0); } return sum; } }
COUNT implements Algebraic interface which looks like this:
public interface Algebraic{ public String getInitial(); public String getIntermed(); public String getFinal(); }
For a function to be algebraic, it needs to implement Algebraic interface that consist of definition of three classes derived from EvalFunc. The contract is that the exec function of the Initial class is called once and is passed the original input tuple. Its output is a tuple that contains partial results. The exec function of the Intermed class can be called zero or more times and takes as its input a tuple that contains partial results produced by the Initial class or by prior invocations of the Intermed class and produces a tuple with another partial result. Finally, the exec function of the Final class is called and produces the final result as a scalar type.
Here's the way to think about this in the Hadoop world. The exec function of the Initial class is invoked once by the map process and produces partial results. The exec function of the Intermed class is invoked once by each combiner invocation (which can happen zero or more times) and also produces partial results. The exec function of the Final class is invoked once by the reducer and produces the final result.
Take a look at the COUNT implementation to see how this is done. Note that the exec function of the Initial and Intermed classes is parameterized with Tuple and the exec of the Final class is parameterized with the real type of the function, which in the case of the COUNT is Long. Also, note that the fully-qualified name of the class needs to be returned from getInitial, getIntermed, and getFinal methods.
Filter Functions
Filter functions are eval functions that return a boolean value. Filter functions can be used anywhere a Boolean expression is appropriate, including the FILTER operator or bincond expression.
The example below uses the IsEmpy builtin filter function to implement joins.
-- inner join A = LOAD 'student_data' AS (name: chararray, age: int, gpa: float); B = LOAD 'voter_data' AS (name: chararray, age: int, registration: chararay, contributions: float); C = COGROUP A BY name, B BY name; D = FILTER C BY not IsEmpty(A); E = FILTER D BY not IsEmpty(B); F = FOREACH E GENERATE flatten(A), flatten(B); DUMP F;
Note that, even if filtering is omitted, the same results will be produced because the foreach results is a cross product and cross products get rid of empty bags. However, doing up-front filtering is more efficient since it reduces the input of the cross product.
-- full outer join A = LOAD 'student_data' AS (name: chararray, age: int, gpa: float); B = LOAD 'voter_data' AS (name: chararray, age: int, registration: chararay, contributions: float); C = COGROUP A BY name, B BY name; D = FOREACH C GENERATE group, flatten((IsEmpty(A) ? null : A)), flatten((IsEmpty(B) ? null : B)); dump D
The implementation of the IsEmpty function looks like this:
import java.io.IOException; import java.util.Map; import org.apache.pig.FilterFunc; import org.apache.pig.backend.executionengine.ExecException; import org.apache.pig.data.DataBag; import org.apache.pig.data.Tuple; import org.apache.pig.data.DataType; import org.apache.pig.impl.util.WrappedIOException; public class IsEmpty extends FilterFunc { public Boolean exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; try { Object values = input.get(0); if (values instanceof DataBag) return ((DataBag)values).size() == 0; else if (values instanceof Map) return ((Map)values).size() == 0; else{ throw new IOException("Cannot test a " + DataType.findTypeName(values) + " for emptiness."); } } catch (ExecException ee) { throw WrappedIOException.wrap("Caught exception processing input row ", ee); } } }
Pig Types
The main thing to know about Pig's type system is that Pig uses native Java types for almost all of its types, as shown in this table.
|
Pig Type |
Java Class |
|
bytearray |
DataByteArray |
|
chararray |
String |
|
int |
Integer |
|
long |
Long |
|
float |
Float |
|
double |
Double |
|
tuple |
Tuple |
|
bag |
DataBag |
|
map |
Map<Object, Object> |
All Pig-specific classes are available here
Tuple and DataBag are different in that they are not concrete classes but rather interfaces. This enables users to extend Pig with their own versions of tuples and bags. As a result, UDFs cannot directly instantiate bags or tuples; they need to go through factory classes: TupleFactory and BagFactory.
The builtin TOKENIZE function shows how bags and tuples are created. A function takes a text string as input and returns a bag of words from the text. (Note that currently Pig bags always contain tuples.)
package org.apache.pig.builtin; import java.io.IOException; import java.util.StringTokenizer; import org.apache.pig.EvalFunc; import org.apache.pig.data.BagFactory; import org.apache.pig.data.DataBag; import org.apache.pig.data.Tuple; import org.apache.pig.data.TupleFactory; public class TOKENIZE extends EvalFunc (DataBag) { TupleFactory mTupleFactory = TupleFactory.getInstance(); BagFactory mBagFactory = BagFactory.getInstance(); public DataBag exec(Tuple input) throws IOException try { DataBag output = mBagFactory.newDefaultBag(); Object o = input.get(0); if (!(o instanceof String)) { throw new IOException("Expected input to be chararray, but got " + o.getClass().getName()); } StringTokenizer tok = new StringTokenizer((String)o, " \",()*", false); while (tok.hasMoreTokens()) output.add(mTupleFactory.newTuple(tok.nextToken())); return output; } catch (ExecException ee) { // error handling goes here } } }
Schema
The latest version of Pig uses type information for validation and performance. It is important for UDFs to participate in type propagation. Until now, our UDFs made no effort to communicate their output schema to Pig. This is because, most of the time, Pig can figure out this information by using Java's Reflection. If your UDF returns a scalar or a map, no work is required. However, if your UDF returns a tuple or a bag (of tuples), it needs to help Pig figure out the structure of the tuple.
If a UDF returns a tuple or a bag and schema information is not provided, Pig assumes that the tuple contains a single field of type bytearray. If this is not the case, then not specifying the schema can cause failures. We look at this next.
Let's assume that we have UDF Swap that, given a tuple with two fields, swaps their order. Let's assume that the UDF does not specify a schema and look at the scripts below:
register myudfs.jar; A = load 'student_data' as (name: chararray, age: int, gpa: float); B = foreach A generate flatten(myudfs.Swap(name, age)), gpa; C = foreach B generate $2; D = limit B 20; dump D;
This script will result in the following error cause by line 4.
java.io.IOException: Out of bound access. Trying to access non-existent column: 2. Schema {bytearray,gpa: float} has 2 column(s).
This is because Pig is only aware of two columns in B while line 4 is requesting the third column of the tuple. (Column indexing in Pig starts with 0.)
The function, including the schema, looks like this:
package myudfs; import java.io.IOException; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; import org.apache.pig.data.TupleFactory; import org.apache.pig.impl.logicalLayer.schema.Schema; import org.apache.pig.data.DataType; public class Swap extends EvalFunc (Tuple) { public Tuple exec(Tuple input) throws IOException { if (input == null || input.size() 2 return null; try{ Tuple output = TupleFactory.getInstance().newTuple(2); output.set(0, input.get(1)); output.set(1, input.get(0)); return output; } catch(Exception e){ System.err.println("Failed to process input; error - " + e.getMessage()); return null; } } public Schema outputSchema(Schema input) { try{ Schema tupleSchema = new Schema(); tupleSchema.add(input.getField(1)); tupleSchema.add(input.getField(0)); return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input),tupleSchema, DataType.TUPLE)); }catch (Exception e){ return null; } } }
The function creates a schema with a single field (of type FieldSchema=) of type =tuple. The name of the field is constructed using the getSchemaName function of the EvalFunc class. The name consists of the name of the UDF function, the first parameter passed to it, and a sequence number to guarantee uniqueness. In the previous script, if you replace dump D; with describe B; , you will see the following output:
B: {myudfs.swap_age_3::age: int,myudfs.swap_age_3::name: chararray,gpa: float}
The second parameter to the FieldSchema constructor is the schema representing this field, which in this case is a tuple with two fields. The third parameter represents the type of the schema, which in this case is a TUPLE. All supported schema types are defined in the org.apache.pig.data.DataType class.
public class DataType { public static final byte UNKNOWN = 0; public static final byte NULL = 1; public static final byte BOOLEAN = 5; // internal use only public static final byte BYTE = 6; // internal use only public static final byte INTEGER = 10; public static final byte LONG = 15; public static final byte FLOAT = 20; public static final byte DOUBLE = 25; public static final byte BYTEARRAY = 50; public static final byte CHARARRAY = 55; public static final byte MAP = 100; public static final byte TUPLE = 110; public static final byte BAG = 120; public static final byte ERROR = -1; // more code here }
You need to import the org.apache.pig.data.DataType class into your code to define schemas. You also need to import the schema class org.apache.pig.impl.logicalLayer.schema.Schema.
The example above shows how to create an output schema for a tuple. Doing this for a bag is very similar. Let's extend the TOKENIZE function to do that:
package org.apache.pig.builtin; import java.io.IOException; import java.util.StringTokenizer; import org.apache.pig.EvalFunc; import org.apache.pig.data.BagFactory; import org.apache.pig.data.DataBag; import org.apache.pig.data.Tuple; import org.apache.pig.data.TupleFactory; import org.apache.pig.impl.logicalLayer.schema.Schema; import org.apache.pig.data.DataType; public class TOKENIZE extends EvalFunc (DataBag) { TupleFactory mTupleFactory = TupleFactory.getInstance(); BagFactory mBagFactory = BagFactory.getInstance(); public DataBag exec(Tuple input) throws IOException { try { DataBag output = mBagFactory.newDefaultBag(); Object o = input.get(0); if (!(o instanceof String)) { throw new IOException("Expected input to be chararray, but got " + o.getClass().getName()); } StringTokenizer tok = new StringTokenizer((String)o, " \",()*", false); while (tok.hasMoreTokens()) output.add(mTupleFactory.newTuple(tok.nextToken())); return output; } catch (ExecException ee) { // error handling goes here } } public Schema outputSchema(Schema input) { try{ Schema bagSchema = new Schema(); bagSchema.add(new Schema.FieldSchema("token", DataType.CHARARRAY)); return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input), bagSchema, DataType.BAG)); }catch (Exception e){ return null; } } }
As you can see, this is very similar to the output schema definition in the Swap function. One difference is that instead of reusing input schema, we create a brand new field schema to represent the tokens stored in the bag. The other difference is that the type of the schema created is BAG (not =TUPLE=).
Error Handling
There are several types of errors that can occur in a UDF:
-
An error that affects a particular row but is not likely to impact other rows. An example of such an error would be a malformed input value or divide by zero problem. A reasonable handling of this situation would be to emit a warning and return a null value. ABS function in the next section demonstrates this approach. The current approach is to write the warning to stderr. Eventually we would like to pass a logger to the UDFs. Note that returning a NULL value only makes sense if the malformed value is of type bytearray. Otherwise the proper type has been already created and should have an appropriate value. If this is not the case, it is an internal error and should cause the system to fail. Both cases can be seen in the implementation of the ABS function in the next section.
-
An error that affects the entire processing but can succeed on retry. An example of such a failure is the inability to open a lookup file because the file could not be found. This could be a temporary environmental issue that can go away on retry. A UDF can signal this to Pig by throwing an IOException as with the case of the ABS function below.
-
An error that affects the entire processing and is not likely to succeed on retry. An example of such a failure is the inability to open a lookup file because of file permission problems. Pig currently does not have a way to handle this case. Hadoop does not have a way to handle this case either. It will be handled the same way as 2 above.
Pig provides a helper class WrappedIOException. The intent here is to allow you to convert any exception into IOException. Its usage can be seen in the UPPER function in our first example.
Function Overloading
Before the type system was available in Pig, all values for the purpose of arithmetic calculations were assumed to be doubles as the safest choice. However, this is not very efficient if the data is actually of type integer or long. (We saw about a 2x slowdown of a query when using double where integer could be used.) Now that Pig supports types we can take advantage of the type information and choose the function that is most efficient for the provided operands.
UDF writers are encouraged to provide type-specific versions of a function if this can result in better performance. On the other hand, we don't want the users of the functions to worry about different functions - the right thing should just happen. Pig allows for this via a function table mechanism as shown in the next example.
This example shows the implementation of the ABS function that returns the absolute value of a numeric value passed to it as input.
import java.io.IOException; import java.util.List; import java.util.ArrayList; import org.apache.pig.EvalFunc; import org.apache.pig.FuncSpec; import org.apache.pig.data.Tuple; import org.apache.pig.impl.logicalLayer.FrontendException; import org.apache.pig.impl.util.WrappedIOException; import org.apache.pig.impl.logicalLayer.schema.Schema; import org.apache.pig.data.DataType; public class ABS extends EvalFunc (Double) { public Double exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; Double d; try{ d = DataType.toDouble(input.get(0)); } catch (NumberFormatException nfe){ System.err.println("Failed to process input; error - " + nfe.getMessage()); return null; } catch (Exception e){ throw WrappedIOException.wrap("Caught exception processing input row ", e); } return Math.abs(d); } public List (FuncSpec) getArgToFuncMapping() throws FrontendException { List (FuncSpec) funcList = new ArrayList (FuncSpec) (); funcList.add(new FuncSpec(this.getClass().getName(), new Schema(new Schema.FieldSchema(null, DataType.BYTEARRAY)))); funcList.add(new FuncSpec(DoubleAbs.class.getName(), new Schema(new Schema.FieldSchema(null, DataType.DOUBLE)))); funcList.add(new FuncSpec(FloatAbs.class.getName(), new Schema(new Schema.FieldSchema(null, DataType.FLOAT)))); funcList.add(new FuncSpec(IntAbs.class.getName(), new Schema(new Schema.FieldSchema(null, DataType.INTEGER)))); funcList.add(new FuncSpec(LongAbs.class.getName(), new Schema(new Schema.FieldSchema(null, DataType.LONG)))); return funcList; } }
The main thing to notice in this example is the getArgToFuncMapping() method. This method returns a list that contains a mapping from the input schema to the class that should be used to handle it. In this example the main class handles the bytearray input and outsources the rest of the work to other classes implemented in separate files in the same package. The example of one such class is below. This class handles integer input values.
import java.io.IOException; import org.apache.pig.impl.util.WrappedIOException; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; public class IntAbs extends EvalFunc (Integer) { public Integer exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; Integer d; try{ d = (Integer)input.get(0); } catch (Exception e){ throw WrappedIOException.wrap("Caught exception processing input row ", e); } return Math.abs(d); } }
A note on error handling. The ABS class covers the case of the bytearray which means the data has not been converted yet to its actual type. This is why a null value is returned when NumberFormatException is encountered. However, the IntAbs function is only called if the data is already of type Integer which means it has already been converted to the real type and bad format has been dealt with. This is why an exception is thrown if the input can't be cast to Integer.
The example above covers a reasonably simple case where the UDF only takes one parameter and there is a separate function for each parameter type. However, this will not always be the case. If Pig can't find an exact match it tries to do a best match. The rule for the best match is to find the most efficient function that can be used safely. This means that Pig must find the function that, for each input parameter, provides the smallest type that is equal to or greater than the input type. The type progression rules are: int=->=long=->=float=->=double.
For instance, let's consider function MAX which is part of the piggybank described later in this document. Given two values, the function returns the larger value. The function table for MAX looks like this:
public List (FuncSpec) getArgToFuncMapping() throws FrontendException { List (FuncSpec) funcList = new ArrayList (FuncSpec) (); Util.addToFunctionList(funcList, IntMax.class.getName(), DataType.INTEGER); Util.addToFunctionList(funcList, DoubleMax.class.getName(), DataType.DOUBLE); Util.addToFunctionList(funcList, FloatMax.class.getName(), DataType.FLOAT); Util.addToFunctionList(funcList, LongMax.class.getName(), DataType.LONG); return funcList; }
The Util.addToFunctionList function is a helper function that adds an entry to the list as the first argument, with the key of the class name passed as the second argument, and the schema containing two fields of the same type as the third argument.
Let's now see how this function can be used in a Pig script:
REGISTER piggybank.jar A = LOAD 'student_data' AS (name: chararray, gpa1: float, gpa2: double); B = FOREACH A GENERATE name, org.apache.pig.piggybank.evaluation.math.MAX(gpa1, gpa2); DUMP B;
In this example, the function gets one parameter of type float and another of type double. The best fit will be the function that takes two double values. Pig makes this choice on the user's behalf by inserting implicit casts for the parameters. Running the script above is equivalent to running the script below:
A = LOAD 'student_data' AS (name: chararray, gpa1: float, gpa2: double); B = FOREACH A GENERATE name, org.apache.pig.piggybank.evaluation.math.MAX((double)gpa1, gpa2); DUMP B;
A special case of the best fit approach is handling data without a schema specified. The type for this data is interpreted as bytearray. Since the type of the data is not known, there is no way to choose a best fit version. The only time a cast is performed is when the function table contains only a single entry. This works well to maintain backward compatibility.
Let's revisit the UPPER function from our first example. As it is written now, it would only work if the data passed to it is of type chararray. To make it work with data whose type is not explicitly set, a function table with a single entry needs to be added:
package myudfs; import java.io.IOException; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; public class UPPER extends EvalFunc (String) { public String exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; try{ String str = (String)input.get(0); return str.toUpperCase(); }catch(Exception e){ System.err.println("WARN: UPPER: failed to process input; error - " + e.getMessage()); return null; } } public List (FuncSpec) getArgToFuncMapping() throws FrontendException { List (FuncSpec) funcList = new ArrayList (FuncSpec) (); funcList.add(new FuncSpec(this.getClass().getName(), new Schema(new Schema.FieldSchema(null, DataType.CHARARRAY)))); return funcList; } }
Now the following script will ran:
-- this is myscript.pig REGISTER myudfs.jar; A = LOAD 'student_data' AS (name, age, gpa); B = FOREACH A GENERATE myudfs.UPPER(name); DUMP B;
Reporting Progress
A challenge of running a large shared system is to make sure system resources are used efficiently. One aspect of this challenge is detecting runaway processes that are no longer making progress. Pig uses a heartbeat mechanism for this purpose. If any of the tasks stops sending a heartbeat, the system assumes that it is dead and kills it.
Most of the time, single-tuple processing within a UDF is very short and does not require a UDF to heartbeat. The same is true for aggregate functions that operate on large bags because bag iteration code takes care of it. However, if you have a function that performs a complex computation that can take an order of minutes to execute, you should add a progress indicator to your code. This is very easy to accomplish. The EvalFunc function provides a progress function that you need to call in your exec method.
For instance, the UPPER function would now look as follows:
public class UPPER extends EvalFunc (String) { public String exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; try{ reporter.progress(); String str = (String)input.get(0); return str.toUpperCase(); }catch(Exception e){ throw WrappedIOException.wrap("Caught exception processing input row ", e); } } }
Load/Store Functions
These user-defined functions control how data goes into Pig and comes out of Pig. Often, the same function handles both input and output but that does not have to be the case.
Load Functions
Every load function needs to implement the LoadFunc interface. An abbreviated version is shown below. The full definition can be seen here.
public interface LoadFunc { public void bindTo(String fileName, BufferedPositionedInputStream is, long offset, long end) throws IOException; public Tuple getNext() throws IOException; // conversion functions public Integer bytesToInteger(byte[] b) throws IOException; public Long bytesToLong(byte[] b) throws IOException; ...... public void fieldsToRead(Schema schema); public Schema determineSchema(String fileName, ExecType execType, DataStorage storage) throws IOException;
The bindTo function is called once by each Pig task before it starts processing data. It is intended to connect the function to its input. It provides the following information:
-
fileName - The name of the file from which the data is read. Not used most of the time
-
is - The input stream from which the data is read. It is already positioned at the place where the function needs to start reading
-
offset - The offset into the stream from which to read. It is equivalent to is.getPosition() and not strictly needed
-
end - The position of the last byte that should be read by the function.
In the Hadoop world, the input data is treated as a continuous stream of bytes. A slicer, discussed in the Advanced Topics section, is used to split the data into chunks with each chunk going to a particular task for processing. This chunk is what bindTo provides to the UDF. Note that unless you use a custom slicer, the default slicer is not aware of tuple boundaries. This means that the chunk you get can start and end in the middle of a particular tuple. One common approach is to skip the first partial tuple and continue past the end position to finish processing a tuple. This is what PigStorage does as the example later in this section shows.
The getNext function reads the input stream and constructs the next tuple. It returns null when it is done with processing and throws an IOException if it fails to process an input tuple.
 This requires further explanation. By default, we would like the loader to do as little per-tuple processing as possible. This is because many tuples can be thrown out during filtering or joins. Also, many fields might not get used because they get projected out. If the data needs to be converted into another form, we would like this conversion to happen as late as possible. The majority of the loaders should return the data as bytearrays and the Pig will request a conversion from bytearray to the actual type when needed. Let's looks at the example below:
A = load 'student_data' using PigStorage() as (name: chararray, age: int, gpa: float); B = filter A by age >25; C = foreach B generate name; dump C;
In this query, only age needs to be converted to its actual type (=int=) right away. name only needs to be converted in the next step of processing where the data is likely to be much smaller. gpa is not used at all and will never need to be converted.
This is the main reason for Pig to separate the reading of the data (which can happen immediately) from the converting of the data (to the right type, which can happen later). For ASCII data, Pig provides Utf8StorageConverter that your loader class can extend and will take care of all the conversion routines. The code for it can be found here.
Note that conversion rutines should return null values for data that can't be converted to the specified type.
Loaders that work with binary data like BinStorage are not going to use this model. Instead, they will produce objects of the appropriate types. However, they might still need to define conversion routines in case some of the fields in a tuple are of type bytearray.
fieldsToRead is reserved for future use and should be left empty.
The determineSchema function must be implemented by loaders that return real data types rather than bytearray fields. Other loaders should just return null. The idea here is that Pig needs to know the actual types it will be getting; Pig will call determineSchema on the client side to get this information. The function is provided as a way to sample the data to determine its schema.
Here is the example of the function implemented by =BinStorage=:
public Schema determineSchema(String fileName, ExecType execType, DataStorage storage) throws IOException { InputStream is = FileLocalizer.open(fileName, execType, storage); bindTo(fileName, new BufferedPositionedInputStream(is), 0, Long.MAX_VALUE); // get the first record from the input file and figure out the schema Tuple t = getNext(); if(t == null) return null; int numFields = t.size(); Schema s = new Schema(); for (int i = 0; i numFields; i++) { try { s.add(DataType.determineFieldSchema(t.get(i))); } catch (Exception e) { throw WrappedIOException.wrap(e); } } return s; }
Note that this approach assumes that the data has a uniform schema. The function needs to make sure that the data it produces conforms to the schema returned by determineSchema, otherwise the processing will fail. This means producing the right number of fields in the tuple (dropping fields or emitting null values if needed) and producing fields of the right type (again emitting null values as needed).
For complete examples, see BinStorage and PigStorage.
Store Functions
All store functions need to implement the StoreFunc interface:
public interface StoreFunc { public abstract void bindTo(OutputStream os) throws IOException; public abstract void putNext(Tuple f) throws IOException; public abstract void finish() throws IOException; }
The bindTo method is called in the beginning of the processing to connect the store function to the output stream it will write to. The putNext method is called for every tuple to be stored and is responsible for writing the tuple into the output. The finish function is called at the end of the processing to do all needed cleanup like flushing the output stream.
Here is an example of a simple store function that writes data as a string returned from the toString function.
public class StringStore implements StoreFunc { OutputStream os; private byte recordDel = (byte)'\n'; public void bindTo(OutputStream os) throws IOException { this.os = os; } public void putNext(Tuple t) throws IOException { os.write((t.toString() + (char)this.recordDel).getBytes("utf8")); } public void finish() throws IOException { os.flush(); } }
Comparison Functions
Comparison UDFs are mostly obsolete now. They were added to the language because, at that time, the ORDER operator had two significant shortcomings. First, it did not allow descending order and, second, it only supported alphanumeric order.
The latest version of Pig solves both of these issues. The pointer to the original documentation is provided here for completeness.
Builtin Functions and Function Repositories
Pig comes with a set of builtin in functions. (NEED LINK) Two main properties differentiate builtin functions from UDFs. First, they don't need to be registered because Pig knows where they are. Second, they don't need to be qualified when used because Pig knows where to find them.
In addition to builtins, Pig hosts a UDF repository called piggybank that allows users to share UDFs that they have written. The details are described in PiggyBank.
Advanced Topics
Function Instantiation
One problem that users run into is when they make assumption about how many times a constructor for their UDF is called. For instance, they might be creating side files in the store function and doing it in the constructor seems like a good idea. The problem with this approach is that in most cases Pig instantiates functions on the client side to, for instance, examine the schema of the data.
Users should not make assumptions about how many times a function is instantiated; instead, they should make their code resilient to multiple instantiations. For instance, they could check if the files exist before creating them.
Schemas
One request from users is to have the ability to examine the input schema of the data before processing the data. For example, they would like to know how to convert an input tuple to a map such that the keys in the map are the names of the input columns. The current answer is that there is now way to do this. This is something we would like to support in the future.
Custom Slicer
Sometimes a LoadFunc needs more control over how input is chopped up or even found.
Here are some scenarios that call for a custom slicer:
-
Input needs to be chopped up differently than on block boundaries. (Perhaps you want every 1M instead of every 128M. Or, you may want to process in big 1G chunks.)
-
Input comes from a source outside of HDFS. (Perhaps you are reading from a database.)
-
There are locality preferences for processing the data that is more than simple HDFS locality.
-
Extra information needs to be passed from the client machine to the LoadFunc instances running remotely.
All of these scenarios are addressed by slicers. There are two parts to the slicing framework: Slicer, the class that creates slices, and Slice, the class that represents a particular piece of the input. Slicing kicks in when Pig sees that the LoadFunc implements the Slicer interface.
Slicer
The slicer has two basic functions: validate input and slice up the input. Both of these methods will be called on the client machine.
public interface Slicer { void validate(DataStorage store, String location) throws IOException; Slice[] slice(DataStorage store, String location) throws IOException; }
Slice
Each slice describes a unit of work and will correspond to a map task in Hadoop.
public interface Slice extends Serializable { String[] getLocations(); void init(DataStorage store) throws IOException; long getStart(); long getLength(); void close() throws IOException; long getPos() throws IOException; float getProgress() throws IOException; boolean next(Tuple value) throws IOException; }
Only one of the methods is used for scheduling: getLocations(). This method allows the implementor to give hints to Pig about where the task should be run. It is only a hint. If things are busy, the task may get scheduled elsewhere.
The rest of the Slice methods are used to read records on the processing nodes. init is called right after the Slice object is deserialized and close is called after the last record has been read. The Pig runtime will read records from the Slice until getPos() exceeds getLength(). Because Slice implements serializable, Slicer can encode information in the Slice that will later be available when the task is run.
Example
This example shows a simple Slicer that gets a count from the input stream and generates that number of Slice s.
public class RangeSlicer implements Slicer, LoadFunc { /** * Expects location to be a Stringified integer, and makes * Integer.parseInt(location) slices. Each slice generates a single value, * its index in the sequence of slices. */ public Slice[] slice (DataStorage store, String location) throws IOException { // Note: validate has already made sure that location is an integer int numslices = Integer.parseInt(location); Slice[] slices = new Slice[numslices]; for (int i = 0; i slices.length; i++) { slices[i] = new SingleValueSlice(i); } return slices; } public void validate(DataStorage store, String location) throws IOException { try { Integer.parseInt(location); } catch (NumberFormatException nfe) { throw new IOException(nfe.getMessage()); } } /** * A Slice that returns a single value from next. */ public static class SingleValueSlice implements Slice { // note this value is set by the Slicer and will get serialized and deserialized at the remote processing node public int val; // since we just have a single value, we can use a boolean rather than a counter private transient boolean read; public SingleValueSlice (int value) { this.val = value; } public void close () throws IOException {} public long getLength () { return 1; } public String[] getLocations () { return new String[0]; } public long getStart() { return 0; } public long getPos () throws IOException { return read ? 1 : 0; } public float getProgress () throws IOException { return read ? 1 : 0; } public void init (DataStorage store) throws IOException {} public boolean next (Tuple value) throws IOException { if (!read) { value.appendField(new DataAtom(val)); read = true; return true; } return false; } private static final long serialVersionUID = 1L; } }
You can invoke the RangeSlicer class with the following Pig Latin statement:
LOAD '27' USING RangeSlicer();


Ocean [aq10], ocean, in The Outer Sea
Last updated Turn #114
Ocean [aq10], ocean, in The Outer Sea
------------------------------------------------------------------------
22: The Savvy Aardvark [ha5r], ship, 2 hulls, 1 rowing port, 6 sails,
22: 6% damaged, arrived from Ocean [ap10], owner:
22: Phantasm [n4b], Imperial Empire, duke, with two workers,
22: eight sailors
23: The Savvy Aardvark [ha5r], ship, 2 hulls, 1 rowing port, 6 sails,
23: 6% damaged, departed for Ocean [aq11].
Routes leaving Ocean:
North, to Ocean [ap10], 6 days
East, to Ocean [aq11], 6 days
South, to Ocean [ar10], 6 days
West, to Ocean [aq09], 6 days


Available Tickets: 50
Available Tickets: 14
Available Tickets: 49
Available Tickets: 50
Available Tickets: 22
Available Tickets: 14
Available Tickets: 49
A family's serenity turns to chaos when a group of doppelgängers begins to terrorize them.
Available Tickets: 50


We Cannot Invalidate Differing Votes: IEC Chairwoman
The recount process has been completed in 17 provinces, is ongoing in ten, and has not yet started in seven.
by Anisa ShaheedHawa Alam Nooristani, Chairwoman of the Independent Election Commission (IEC), said on Wednesday that the IEC cannot invalidate about 300,000 differing votes before the recount and audit process ends.
Nooristani said those votes should be assessed in the recounting centers.
“We and our technical colleagues met with the representatives of candidates, and they (representatives) insist…that the commission invalidates the votes, but vote invalidation cannot happen through the IEC, we must go to the area and examine the boxes and then whatever is invalid we will not count-- invalid is invalid,” said Nooristani.
A number of electoral teams, especially Abdullah Abdullah’s, say that the 300,000 disputed votes have no biometric marks and must be invalided.
As a result, the election teams boycotted the recount process, and according to the IEC, the election teams have prevented recounts in seven provinces.
The IEC says about 8,000 votes cast in the presidential election should be recounted.
The recount process resumed after a four-day halt on Sunday.
According to the IEC, the recount process has been completed in 17 provinces, is ongoing in ten provinces, and has not started in seven provinces.
“We will announce the results base on the IEC timetable,” said Aurangzeb, a member of the Independent Election Commission.
“They have lost the ability to control the process. The whole work is done by the IEC secretariat and secretariat chief and provincial directors,” said Shahab Hakimi, a presidential candidate.
Based on the IEC timetable, the preliminary results should have been announced by October 19, but nearly two months have passed and still the preliminary results have not been announced.
Aurangzeb said: “Once the vote recounting process ends we will announce the results,” said Aurangzeb.


Hi everyone,
I am a proud owner of an emonTx Shield V2, and Arduino Uno and 4 CT sensors(SCT-013-000). I have setup 2 wires to power a 100W light bulb and put 1 CT sensor over one of the wires.
I've loaded Arduino with the SingleCT example code, turn on the light, open the serial and it shows me 0.00 :(
I then moved the sensor in the next jack CT2, now it returns the Irms=0.28 but it does that even if I pull out the CT from the jack or I turn off the light.
Then I tried CT3 it says 0.00 ... CT4 returns 0.28 just like CT2 it doesn't matter if light is on or off :(
Do you think something might be wrong while the soldering?
Do I need to do anything about the jumpers if I don't have RF module? One of the jumpers is set to Uno, the other one is missing :(
Any help is much appreciated.
Thank you
Re: I need some help to get started
What I/O pins are you using in the sketch? Remember the emonTx and the emonTx SHIELD are two different beasts and they use different I/O pins for different functions. So you must use sketches written for the Shield with the Shield, or change the I/O pins around.
The 0.28 A you are reading is noise, getting into the ADC from the digital circuits inside the processor.
Re: I need some help to get started
Thanks for the hints. Problem solved!
Re: I need some help to get started
Hi again,
I have another issue to which i would appreciate your help.
If I run the arduino powered just by the USB I get a fairly accurate reading(40W for a 20W light bulb), but if power the arduino by a 9V 2Amp adaptor, I get a reading of 2000W for the 20W light bulb.
What could be wrong? Adaptor broken, or everything is normal and I just need to calibrate the sensor based on the power source that I have?
Thanks
Re: I need some help to get started
That shouldn't be happening. Am I right in thinking that you are measuring current only?
The first thing to check is the voltage supplied to your Arduino to check that the power supply regulators are working. The processor should be running on 5 V, which you should get on the points marked "5V" on the shield. I don't know the details of Arduino boards so I can't help much.
Secondly, even if the supply isn't exactly 5 V, you still shouldn't read 100 × the correct current, so something is badly wrong. The power is calculated assuming a voltage of 230 V, if yours is different then the power will be wrong, but again not 100 ×.
Re: I need some help to get started
Yeah, measuring current only. I'll check the shield 5V pin if its indeed 5V. Thanks


Pro_Pack : skining¶
ProPack skinning management and file format for Maya
DEPRICATED AND REPLACED WITH THE NEW “skinning” module
>>> # import statement for the module via the r9pro decompiler
>>> from Red9.pro_pack import r9pro
>>> r9skinning = r9pro.r9import('r9skinning')
Main Classes
Selectionfilter () |
|
Skincluster ([skincluster]) |
|
Sknfile () |
|
Shapelayout (ui, shape, *args, **kws) |
|
R9sknui ([parent]) |
-
class
Hi_selection
(instance, data, jnt_roots=[], load_data=False, checkdata=True, ignore_selection=False, parent=None, *args, **kws)¶ Bases:
PySide2.QtWidgets.QMainWindow
-
set_selection
(root=None)¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
Jnt_remap
(parent=None, Skn_widget=None)¶ Bases:
PySide2.QtWidgets.QMainWindow
,Ui_mw_aditive_joint_selector
-
data
¶
-
joints
¶
-
new_jnt
¶
-
old_jnt
¶
-
old_joints
¶
-
refresh
()¶
-
remap
(old_jnt, new_jnt, get_data=True)¶
-
select_joint_new
()¶
-
select_joint_old
()¶
-
select_vertex
(list_to_use='old')¶
-
set_new
()¶
-
set_old
()¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
R9Skn_widget
(ui, shape, data, *args, **kws)¶ Bases:
PySide2.QtWidgets.QHBoxLayout
-
data
¶
-
debug
(fix=False)¶
-
deleteLater
(*args, **kwargs)¶
-
find_joints
(data, func=None)¶
-
load
(data=None)¶
-
remap
(newshape=None)¶
-
remap_jnt
()¶
-
save
()¶
-
select_joints
()¶
-
select_shape
()¶
-
select_skincluster
()¶
-
select_vertex
()¶
-
shape_exist
()¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
R9skn
(shape=None)¶ Bases:
object
-
check_maximun_infuences
(data=None, select=True, fix=False, logit=True)¶
-
check_missing_joints_on_cluster
(**kwargs)¶
-
check_missing_joints_on_scene
(**kwargs)¶
-
check_negative_weights
(data=None, select=True, fix=False, logit=True)¶
-
check_set_data
(data, vrtxids)¶
-
check_weight_values
(data=None, select=True, fix=False, logit=True)¶
-
create_skincluster
(**kwargs)¶
-
fix_weights
(fix=False, select=True)¶
-
get_all_weights
()¶ get the skiing weights of all vertices of the shape
Returns: dictionary of dictionaries with the numeric id of the vertex as a key and a dictionary as value containing skinning values, the joint influence name as key and weight value as value
-
get_data
(debug=False, ignore_selection=False, namespace=True)¶ main call to collect the skinning data and return a dict
-
get_joints
()¶ get joint that are part of skin cluster :return: list of strings
-
get_joints_from_root
(root, data)¶ compare joints in data against joint hierarchy and return matching joints
-
get_root_joints_from_data
(data)¶ analyze data and find root joints in the scene :param data: dict skin data :return: list of joints
-
get_selected_vrtx_ids
()¶ Returns: a string list of numeric ids of currently selected vertex
-
get_vrtx_weights
(vrtxid=None)¶ analise a given or selected vertex and return a dictionary wit the joints and influence values
Parameters: vrtxid – numeric id of the vertex Returns: dictionary with the weight values of the vertex keys are the joint influence name and value the is the value of influence
-
get_vrtxs_weights
(vrtx_ids=None)¶ analise a given list or selected vertexes and return a dictionary of dictionaries wit the joints and influence values
Parameters: vrtxids – list of numeric ids of vertexes Returns: dictionary of dictionaries with the numeric id of the vertex as a key and a dictionary as value containing skinning values, the joint influence name as key and weight value as value
-
idfromattr
(attr)¶ split a guiven string attribute to return the numeric id as string :param attr: string attr :return: string numeric id
-
jntids
(key='id')¶ return a dictionary of joint ids that belong to the skin cluster, keyword key can be use to specify the way the data is return if key=’id’ id will be the keys of the dictionary and joint name the value if key=’joint’ the joint name will be the key and the id the value
Parameters: key – string to define which value is the key of the dictionary, valid parameters are ‘id’ and ‘joint’ Returns: dictionary containing ids and joint names the order of keys and values can be change by the use of key keyword
-
load
(data, checkdata=True, ignore_selection=False, root=None)¶
-
ls
()¶
-
nameSpace
()¶
-
root_joints
(data=None)¶ Returns: a list of root joints from skin cluster
-
select_vrtx
(vrtx_id)¶ select given vertex id :param vrtx_id: int vertex id
-
select_vrtxs
(vrtx_ids)¶ select given vertex id :param vrtx_ids: list of ints
-
set_weights
(weights, vrtx_ids=False, checkdata=True)¶
-
skincluster
¶ Returns: string skinCluster node name attached to the shape
-
vrtxids
()¶ return a list of vertex ids that belong to the skin cluster :return: list of string of vrtx ids
-
-
class
R9skn_file
¶ Bases:
object
-
get_file_path
(action='save')¶
-
get_scene_data
(debug=False)¶
-
get_skns
(selection=False)¶
-
namebaseonscene
()¶
-
read_skin_file
(file_path=None, namespace=None)¶
-
save_file
(file_path=None, debug_fix=True, skip_ui=False)¶
-
-
class
R9sknui
(parent=None)¶ Bases:
PySide2.QtWidgets.QMainWindow
,Ui_mw_r9skn
-
browse
(sknfile=None)¶
-
clearshapesinsknfile
()¶
-
getjointsindata
(data)¶
-
getverticesindata
(data)¶
-
load
()¶
-
save
(debug=True)¶
-
setshapesinsknfile
()¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
R9sknui2
(parent=None)¶ Bases:
PySide2.QtWidgets.QMainWindow
,Ui_mw_r9skn
-
add_widget
(shape, data=None)¶
-
clear_ui
()¶
-
closeEvent
(event)¶
-
debug
(fix=False)¶
-
load
()¶
-
map_namespace
()¶
-
open_file
(file_path=None, namespace=None)¶
-
save
()¶
-
set_sknfile_actions
()¶
-
set_widgets
(selection=False)¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
Root_select
(func, jnt_roots=[], parent=None, *args, **kws)¶ Bases:
PySide2.QtWidgets.QMainWindow
-
set_selection
(root=None)¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
Shapelayout
(ui, shape, *args, **kws)¶ Bases:
PySide2.QtWidgets.QHBoxLayout
-
deleteLater
(*args, **kwargs)¶
-
load
()¶
-
remap
()¶
-
selectjoints
()¶
-
selectshape
()¶
-
selectvertives
()¶
-
shapeexist
()¶
-
staticMetaObject
= <PySide2.QtCore.QMetaObject object>¶
-
-
class
Skincluster
(skincluster=None)¶ Bases:
object
-
checkmaximuninfuences
(data=None, select=True, fix=False, logit=True)¶
-
checknegativeweights
(data=None, select=True, fix=False, logit=True)¶
-
checkweightvalues
(data=None, select=True, fix=False, logit=True)¶
-
copy_paste
()¶
-
copyweight
(vrtxid=None)¶
-
debug
(fix=False, select=True, logit=True)¶
-
get_verts_ordered_selection
()¶
-
getallweights
()¶
-
getselectedvrtxids
()¶
-
getshape
()¶
-
getvrtxsweights
(vrtxids=None)¶
-
getvrtxweights
(vrtxid=None)¶
-
idfromattr
(attr)¶
-
jntids
(key='id')¶
-
load_copy_paste_history
(skncph=None)¶
-
loadweights
(weights, vrtxids=False)¶
-
namebaseonscene
()¶
-
pasteweight
(vrtxids=None)¶
-
run_history
()¶
-
select
()¶
-
select_history_verts
()¶
-
selectvrtx
(vrtxid)¶
-
selectvrtxs
(vrtxids)¶
-
setskinclusterfromobject
(node=None)¶
-
testloadedata
(data, vrtxids)¶
-
vrtxids
()¶
-
write_copy_paste_history
()¶
-
-
class
Sknfile
¶ Bases:
object
-
debug
(fix=False, select=False, logit=True)¶
-
getsknfile
()¶ get skn file location on pc
-
loadWeights
(sknfile=None)¶
-
namebaseonscene
()¶
-
namebaseonshape
(shape)¶
-
readsknfile
(sknfile=None)¶ reads a given skin file :param sknfile: path to .skn file :return skin data as a dictionary
-
saveWeights
(sknfile=None, debug=True)¶
-
setsknfile
(sugestedname)¶ set skn file location on pc
-
writesknfile
(data=None, sknfile=None)¶
-
-
delete_all_dagposes
()¶
-
delete_all_skin_clusters
()¶
-
get_meshes
(selection=False)¶ Parameters: selection – bool if true it will just filter selected meshes Returns: string list with all the meshes in the scene or filter by selection
-
jointhiroot
(joint)¶ get the name of the root joint in the hierarchy :param joint: name of a joint in the hierarchy :return: name of the root joint in the hierarchy
-
load_skn_file
(filepath=None, mesh=None, vc=False, root=None)¶ load a skin file onto a given mesh or all meshes in the scene.
Parameters: - filepath – skin file
- mesh – given mesh / meshes to load if matched in the skin file data
- root – string root joint of the skeleton hierarchy
Parma vc: match by wildcard based on vertex count of the mesh itself
-
openui
()¶
-
read_file
(filepath, namespace=None, joints=True, shapes=True)¶ Read skin file, if namespace is provided all the data file will be mapped to the given namespace
Parameters: - filepath – skin file path
- namespace – namespace to use for re-mapping the file
Returns: dict of skin file data
-
remove_namespace_on_data
(data, file_data=True)¶
-
select_all_dagposes
()¶
-
select_all_skin_clusters
()¶


Still remember those days
MEETING
THE MOMENT,
TOGETHER
Registration Portal
Alumni Registration
We have started ICC-CET Alumni Forum in the first alumni meet held on 9th July 2016 @Calicut, To coordinate various activities among us.This forum will follow the principles of ICC-CET Trivandrum. Our main objective is to motivate and coordinate our younger brothers and sisters in academic upliftment as well as career development.
Book Collection Drive
An ICC CET Alumni initiative to enrich the IC Library with help of its Alumni. We are collecting used guide books for competitive exams like GATE,CAT,IES,UPSC CSE etc, which can be donated to IC library. Donated books will be made available for the present students and will be benefited in coming years In sha Allah.
Apply for Alumni Scolarship
Apply for Islamic Cultural Center (ICC) Alumni interest-free loan Scholarship Scheme 2020-21 for better education of our brothers
APPLY NOWContribute to Alumni Scloarship
This forum is intended to keep a record of the contributing people of our relief amount. In sha Allah we will maintain the secrecy of the information .
Contribute


In the past, I worked under someone having the quality of compassionate leadership. He had all the three components of cognitive ("I understand you"), affective ("I feel for you"), and motivational ("I want to help you"). When he was just a Senior Engineer, I didn't see the components from him. As an Intermediate Engineer, I just thought that he was a weak and soft spoken person. And I doubt that he had the capability to manage a team under his soft shell when he was promoted to manage a team. (To put you into perspective, the other managers were like fierce tigers and dragons that won't let go the preys in their mouth. They are the perfect example of not compassionate leadership)
There was a time my team was lacking a manager. The person was assigned to be the manager in my team. When the news broke up that the person will be our manager, one of my colleagues rose his thumb and said that: "Good news!". Well, I doubted his excitement. Anyway, the colleague was someone I respected. So, I took his words and felt hopeful that we will be getting a good manager. The immediate effect I noticed when working for the new manager can be described in one word: "Comfortable". The next thing, I felt that I could trust him and ask for his help when I was in trouble with my work. As we continue working together, I finally can see the secret power behind his soft shell. In one of the projects, the new software tool that I worked on was under the risk of not meeting the schedule, putting the whole project at risk. The first thing he did, was to understand about my situation and the reason of cannot deliver on time. He felt my frustration on the situation and he knew that I was taking blame for the delay in the project. Then, he used his negotiation skills to help me to get agreement from my stakeholders to prioritize the tasks so that some tasks could be delivered after the critical milestone. He helped me went through my career crisis.
My colleague once told me a story regarding what he observed. There was a time when a senior manager meeting with a group of managers to discuss some issues. Whenever the other managers expressing their opinions, the senior manager could not agree with them. But when it was my compassionate manager speaking, the senior manager listen really carefully and agree to what he said.
After doing multiplying goodness meditation, the goodness felt abundant. Multiplying goodness also make me feeling happy and useful (because of contributing goodness).
It was not easy to do. It could lead to depression if perspective was wrong (need elevated perspective), or transformation was omitted (need to let dark cloud disperse and transform to ray of lights). After completing the process, I could felt myself at ease, confidence and capable of giving kindness. (May be it was a coincidence, my partner came to hug me after 5 minutes of finishing Tonglen meditation. I secretly used her as a practice target)
SCARF = Status, Certainty, Autonomy, Relatedness, Fairness
Status: I could feel the status reward when my mastery of HTML, CSS and Javascript grew. I knew that I accidentally triggered status threat when I told Harrison that the coding task was not challenging and he promised me that I will feel sorry in phase-1.
Certainty: Let take the calculator project that I worked on as an example. I knew that the original calculator project was not difficulty for me (certainty), so I decided to stretch further to work on a more complex version of the calculator (uncertainty). After started working on the complex version of the calculator, I found that the project was more difficult than my original estimation (uncertainty). I split the work to different stages so that I could conquer them one step at a time (certainty). As more features were finished (certainty), I had less worry about the undone features (uncertainty). Eventually, I finished everything, and finally felt a sense of peace within me (certainty).
Autonomy: I would like to finish the tasks on time, every week. However, there usually got something happened in the real life that need my immediate attention. I felt helpless and stress when the thing in real life make me behind the schedule.
Relatedness: The first weekly check-in were both excited and scary. Everyone was a stranger to me (foe). I didn't feel comfortable to speak to so many strangers at once. As the strangers started sharing their stories, personal and technical, I seems to know them better (friend), and felt more at ease.
Fairness: Everyone in the bootcamp will be going through the same syllabus (fairness). But I have advantages over them due to my previous background (unfairness). I could sense that some of them need help, so I took on mentoring role to help those who needed help. (fairness)
Understanding SCARF is an important first step. It should help me to be a better team player toward my peers during phase-1 bootcamp.
Erm, let me try to talk to Rohan about giving me 50% rebate on the bootcamp course fee. Well, may be not. The author of 'Search Inside Yourself' advise that “Sometimes, the right thing to do is not to raise the issue at all.” Sorry for the disappointment, please tune your radio to 94.1FM (for Wellington area) to listen to THE BREEZE to get some fresh air.


{% load staticfiles %}
django-export-csv
is a reusable Django application for which provides generic views for downloading CSV files.
pip install git+https://github.com/narenchoudhary/django-export-csv#egg=django-export-csv
Add 'export_csv'
to INSTALLED_APPS
settings of the project.
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
...
...
'export_csv',
]
Subclass ExportCSV
in views.py file and provide model attribute.
from export_csv.views import ExportCSV
from .models import Account
class AccountCSV(ExportCSV):
"""View for creating and rendering CSV of all Account model instances."""
model = Customer
In urls.py, add url pointing to the view class AccountCSV
declared above.
from .views import AccountCSV
urlpatterns = [
(r'^account_csv/$', AccountCSV.as_view(), name='account-csv'),
]
That's it. It will render a CSV file containing all the fields of all the instances of Account
model.
| Transacton ID | Account ID | Exchange | Fraudulent | Date | Deposit/Withdrawal |
|---|---|---|---|---|---|
| {{transaction.transaction_id}} | {{transaction.account.account_no}} | {{transaction.exchange}} | {{transaction.is_fraudulent | yesno }} | {{transaction.transaction_date}} | {{transaction.deposit_or_withdrawal}} |
Following code is directly taken from
example.app.views.py
file of the app.
import csv
from export_csv.views import ExportCSV
from .models import Transaction
class TransactionCSV(ExportCSV):
# provide model whose queryset are to be exported to CSV
model = Transaction
# provide names of fields to be included in CSV
field_names = ['account', 'transaction_id', 'transaction_date']
# provide custom filename
filename = 'transactions_csv_filename.csv'
def get_csv_writer_kwargs(self, **kwargs):
return dict(quoting=csv.QUOTE_ALL, delimiter=b' ', quotechar=b'|')
def clean_transaction_id(self, value):
return str(value).lower()
| Account No | Creation Date | Account Owner | Account Balance | Last Withdrawn | Last Deposited | Transaction Count |
|---|---|---|---|---|---|---|
| {{account.account_no}} | {{account.creation_date}} | {{account.owner}} | {{account.balance}} | {{account.last_withdrawn}} | {{account.last_deposited}} | {{account.transaction_count}} |
Following code is directly taken from
example.app.views.py
file of the app.
import csv
from export_csv.views import ExportCSV
from .models import Account
class AccountCSV(ExportCSV):
"""View for creating and rendering CSV of Account instances returned by
custom queryset.
"""
# use this filename for rendering CSV
filename = 'rich_account_list.csv'
def get_queryset(self):
"""Write to CSV only those Account instances whose balance is greater
than 600000."""
return Account.objects.filter(balance__gt=600000)
def get_field_names(self):
"""Write to CSV only account_no and owner fields."""
return ['account_no', 'owner']
| Name | Address | Active | Last Updated |
|---|---|---|---|
| {{customer.name}} | {{customer.address | linebreaksbr }} | {{customer.is_active | yesno }} | {{customer.last_updated }} |
Following code is directly taken from
example.app.views.py
file of the app.
import csv
from export_csv.views import ExportCSV
from .models import Customer
class CustomerCSV(ExportCSV):
"""View for creating and rendering CSV of all Customer model instances."""
model = Customer


{% extends "base/base.html" %} {% block title %}Delete article {{ article.title }}{% endblock %} {% block content %}
Are you sure you want to delete this article?
No, take me back.


| PostgreSQL 9.0.10 Documentation | ||||
|---|---|---|---|---|
| Prev | Up | Appendix I. Documentation | Next |
Reference pages should follow a standard layout. This allows users to find the desired information more quickly, and it also encourages writers to document all relevant aspects of a command. Consistency is not only desired among PostgreSQL reference pages, but also with reference pages provided by the operating system and other packages. Hence the following guidelines have been developed. They are for the most part consistent with similar guidelines established by various operating systems.
Reference pages that describe executable commands should contain the following sections, in this order. Sections that do not apply can be omitted. Additional top-level sections should only be used in special circumstances; often that information belongs in the "Usage" section.
This section is generated automatically. It contains the command name and a half-sentence summary of its functionality.
This section contains the syntax diagram of the command. The synopsis should normally not list each command-line option; that is done below. Instead, list the major components of the command line, such as where input and output files go.
Several paragraphs explaining what the command does.
A list describing each command-line option. If there are a lot of options, subsections can be used.
If the program uses 0 for success and non-zero for failure, then you do not need to document it. If there is a meaning behind the different non-zero exit codes, list them here.
Describe any sublanguage or run-time interface of the program. If the program is not interactive, this section can usually be omitted. Otherwise, this section is a catch-all for describing run-time features. Use subsections if appropriate.
List all environment variables that the program might use. Try to be complete; even seemingly trivial variables like SHELL might be of interest to the user.
List any files that the program might access implicitly. That is, do not list input and output files that were specified on the command line, but list configuration files, etc.
Explain any unusual output that the program might create. Refrain from listing every possible error message. This is a lot of work and has little use in practice. But if, say, the error messages have a standard format that the user can parse, this would be the place to explain it.
Anything that doesn't fit elsewhere, but in particular bugs, implementation flaws, security considerations, compatibility issues.
Examples
If there were some major milestones in the history of the program, they might be listed here. Usually, this section can be omitted.
Cross-references, listed in the following order: other PostgreSQL command reference pages, PostgreSQL SQL command reference pages, citation of PostgreSQL manuals, other reference pages (e.g., operating system, other packages), other documentation. Items in the same group are listed alphabetically.
Reference pages describing SQL commands should contain the following sections: Name, Synopsis, Description, Parameters, Outputs, Notes, Examples, Compatibility, History, See Also. The Parameters section is like the Options section, but there is more freedom about which clauses of the command can be listed. The Outputs section is only needed if the command returns something other than a default command-completion tag. The Compatibility section should explain to what extent this command conforms to the SQL standard(s), or to which other database system it is compatible. The See Also section of SQL commands should list SQL commands before cross-references to programs.


--- layout: default title: Sponsor Code for San Francisco description: Support Code for San Francisco! ---
We believe in civic innovation, open data, and open source collaboration.
Code for San Francisco is a community searching for technological solutions to social challenges. We are an all volunteer group of technologists, designers, coders, data scientists, and engaged citizens. Our weekly hack night is open to anyone interested in making San Francisco a better city. See what a typical night looks like.
We depend on the generous support of local organizations to help us run events and for our projects to accomplish their goals. When you donate to Code for San Francisco, you’re supporting the creation of civic technology solutions for the local community.
If you are interested in supporting our organization, please reach out via<EMAIL_ADDRESS>Donations are managed through Code for America, a 501(c)(3) nonprofit corporation and your donation is tax deductible.
Our weekly hack nights offer the San Francisco civic tech community a safe space for experimentation, learning and practicing skills, taking chances, and asking questions, all to effect and encourage positive change. Whenever possible, we proactively reach out to those who are left out, excluded, ignored, or discriminated against. We fail without their involvement.
Can’t make it to hack night? Our projects are organized through Slack and Github. Remote volunteers are welcome.
In addition to our weekly hack nights which draw ~50 participants, we also hold special events throughout the year including:
Launched in October 2016, the Adopt a Drain Program enables San Francisco residents to "adopt" one of the City's 25,000 storm drains, keeping it free of debris. While helping to reduce localized flooding, the program also fosters community engagement by encouraging San Franciscans to take an active role in improving their neighborhood.
It has since been adopted by the San Francisco Public Utilities Commission.
Our working group exists to promote learning and mutual improvement among its members, while also providing support and advisory services for project teams and the Brigade as a whole.
We believe this group is important because it helps reinforce user-centered design principles that are essential for the delivery of effective digital products and services.
Your generous gift will support our meetups and events throughout the year:
National Day of Civic Hacking 2017
Sponsored food from CodeAcross 2016
Sponsor one of our hack nights. Your name and logo will appear on the event page and any materials for the evening, recognition via our Twitter and Facebook the evening of. If a representative is available for the sponsored evening, they will be invited to speak during the opening remarks.
Sponsor hack night for a month! Your name and logo will appear on the event pages and any materials for 4 weeks worth of hack nights as well as recognition through our Twitter and Facebook throughout that time. Logo placement on our homepage at the "Official sponsor" level for 12 months following the donation. If a representative is available for one of the sponsored evenings, they will be invited to speak during the opening remarks.
Invitation to give a presentation OR to conduct a workshop during of our weekly hack nights. Sustaining sponsor gifts will be covered on our blog, in our newsletter, and on our Twitter and Facebook. Logo placement on our homepage at the "Sustaining sponsor" level and on our Meetup group for 12 months following the donation.
Major sponsors are invited to promote events, recruit volunteers, and coordinate with the Core Team on other avenues of partnership on a continuing basis. Major sponsor gifts will be covered on our blog, in our newsletter, and on our Twitter and Facebook.Includes logo placement on home page at the "Major sponsor" level and Meetup group for 12 months following the donation.
Please reach out via fundraising@codeforsanfrancisco.org.


i-Innovate has a systematic innovation process laid out for the students to follow and learn through an experiential product or service building. This platform exposes them to test their hypothesis of entrepreneurship by validating the idea. Affordability, Sustainability, Scalability, Universal, Rapid, Excellence, and Distinctive are the dimensions that are weighted. The i-Innovate platform also guides the students in form of mentorship to deserving top ideas. So what are you waiting for, this is the right time to register for this adventure!!!
Awards
Platinum
10 Lakhs
25k for Mentor
Gold
5 Lakhs
15k for Mentor
Silver
2.5 Lakhs X 2
10k for each Mentor
Most Popular
1 Lakh
Benefits
Solve problems related to Mobility and Energy
IP ownership with Students
Employment opportunities on idea submission
Incubation opportunities
Project Timeline
6th March - 30th Sept 2021
Registration, Idea submission, Prototyping & Evaluation
1st July - 31st December 2021
Guidance to Top 100 Teams
1st January - February 2022
Finalists Selection and Grand Finale
Hall Of Fame
KPIT Sparkle 2020
In its sixth year now, the theme for KPIT Sparkle 2020 was Mobility & Energy for the future.
The event was held on 29th February and 1st March 2020 at the Pimpri Chinchwad College of Engineering (PCCOE), Akurdi – Pune. This year the contest attracted participation from 1000+ colleges in 25 States & 4 Union Territories of India, representing an astonishing 20000+ students through 3000+ projects. Highlights for this year was the increased participation from premier institutes (IITs – 17, NITs – 21).
Team Detox from Army Institute of Technology, Pune, won the gold award of Rs. 5,00,000.
Team Vortex from Indian Institute of Technology, Kharagpur won the silver award for designing a vortex energy generator.
Team ZincAir from Jawaharlal Nehru Centre for Advanced Scientific Research, Bengaluru won the silver award for developing Eco-friendly Batteries for Electric Vehicles.
Team Bio from ST. XAVIERS CATHOLIC COLLEGE OF ENGINEERING, Kanyakumari won the Most Popular Award for designing a Microalgal Hydrogen Production using Pollutant Consuming Photobioreactor. They won by acquiring 13190 votes from a total of 74987.
KPIT Sparkle 2019
The fifth edition of KPIT Sparkle received more than 2000 ideas from around 22,000 engineering and science students across India. The 30 best teams presented prototypes of their solutions to the jury and incubation partners at the grand finale at a public exhibition, held at the premises of the event's academic partner, Pimpri Chinchwad College of Engineering (PCCOE), Pune.
Team Aqua from Army Institute of Technology, Pune, won the platinum award of Rs. 10,00,000 for developing a vertical axis water turbine (VAWT), which is 24% more efficient than a conventional system. The turbine can be used for both small and large-scale energy production and in all flow conditions.
Team Gravity from Bharath Institute of Higher Education and Research, Chennai, for designing a scalable plant-based microbial fuel cell technology. The solution has the capacity to generate electricity and eliminate dependence on climate for renewable energy production.
Team AutoTrac from College of Engineering, Indore, for designing a driverless tractor, which saves labour cost, time and manual labour.
Team Drivetrain from Jayawantrao Sawant College of Engineering, Pune, won the Most Popular Award for designing a novel transmission system (single stage gear box) to increase the load carrying capacity and gradeability of existing e-vehicles. The team garnered online votes in excess of 11,000 for their project.
KPIT Sparkle 2018
This was the fourth year of KPIT Sparkle a national design and development innovation contest for science and engineering students. The theme of the competition in this edition was 'Next Generation Energy and Transportation Solutions'. This year we received over 12,000 registrations from more than 600 engineering and science colleges, highlights for this year was the increased participation from premier institutes (IITs, NITs) from 9 to 28. We also saw an increase participation 63 of top 100 institutes across India, the list as published by the MHRD (NIRF rankings).
Team Electrodes from Indian Institute of Technology, Kharagpur won the Platinum award of Rs. 10 lakhs for their project Bacteria Powered Battery
Team SmartGears from SAHYADRI COLLEGE OF ENGINEERING & MANAGEMENT, Mangalore won the Gold award of Rs. 5 lakhs for their project Smart Helmet
Team NanoKnocks from PUNE VIDYARTHI GRIHA'S COLLEGE OF ENGINEERING, Pune won the Silver award of Rs. 2.5 lakhs for their project Organic Nano Material for cleaning Oil Spills.
Team Thermo from PAD. DR. D. Y. PATIL INSTITUTE OF ENGINEERING, MANAGEMENT & RESEARCH and PIMPRI CHINCHWAD COLLEGE OF ENGINEERING, Pune won the Silver award of Rs. 2.5 lakhs for their project Converting Waste Heat into electricity
Team Path Predictor from NATIONAL INSTITUTE OF TECHNOLOGY, HAMIRPUR won the Most Popular Award of Rs. 1 lakh for their project Predictive Path Analyzer for Priority vehicles. This team received highest no of votes across India.
KPIT Sparkle 2017
The 3rd edition of the design and development innovation contest, in collaboration with College of Engineering, Pune (COEP), received 1,500 ideas on the theme of Smart Cities from over 10,500 engineering and science students across India
Team AlphaGears from Mangalore Institute of Technology and Engineering won the Platinum prize of Rs. 10 lakhs for their project on frictionless gear transmission system for vehicles using Polymagnets. This solution will overcome the drawbacks of present day mechanical gear system.
Team Robo-X-Prime from R. C. Patel Institute of Technology, Shirpur, Maharashtra won the Gold prize for developing a robot for Power Transmission Line Inspection.
Team Robo-Climb from College of Engineering, Pune (COEP) for their autonomous vehicle solution called Wall Climbing Robot, which can be used for this robot for transportation of goods as well as passengers over the distance.
Team Traffic Eye from Birla Institute of Technology and Science (BITS), Pilani for their project Traffic Eye to check traffic violations.
The Most Popular project award went to Team Humanoids from Kalinga Institute of Industrial Technology, Bhubaneswar for their project on Exhaust Gas Emission Control device using Plasma Technology which results in less electric power consumption with no harmful gases discharged in the environment.
KPIT Sparkle 2016
KPIT Announces Winners of KPIT Sparkle 2016, All India Science and Engineering Students Contest
Top shortlisted teams from engineering and science colleges across India presented innovative,
robust solutions to address energy and transportation challenges.
Team Assist from The Indian Institute of Technology, Kharagpur won the Gold award of INR 5 Lakhs for their "i-Bike" project. The bicycle model can drive itself autonomously and also be driven manually, as and when required, in an urban bicycle-sharing system to achieve in last-mile transportation.
Team Explorers from Nehru College of Engineering and Research Centre, Pambady, Kerala for their project IC engine efficiency increase without compromising present engine design.
Team Foxtrot Geeks from Hindusthan College of Engineering and Technology, Coimbatore, for their project on virtual speed breaker.
Team BIA from Indian Institute of Technology, Kharagpur, for the project on green and facile synthesis of novel supercapacitor electrode material using JUTE fibres as precursor.
RMR M.S. Ramaiah Institute of Technology, Bangalore for the project on Remote Road Quality Assessment and Analysis.
Team Varenya from College of Engineering, Pune for their for developing a system to increase the efficiency and effectiveness of solar water heating system.
Team Trikeracers from College of Engineering, Pune for their project on reverse pedal braking system.
KPIT Sparkle 2015
The first edition of KPIT Sparkle received more than 400 ideas from around 800 engineering and science students across India. The 60 best teams presented prototypes of their solutions to the jury at the grand finale at a public exhibition, held at the premises of the KPIT Technologies, Pune.
Jainendra Mishra from Siddaganga Institute of Technology, Tumkur, Karnataka won the grand prize (Platinum award) of INR 10 lakhs for his innovation. He developed a solution that was aimed at increasing efficiency of hybrid and electric vehicles, which run on lithium-polymer battery pack, by using a novel circuitry grid using SANDY (Switching algorithm ANd cell DYnamics) algorithm.
The team of Suraj Aru, Shubham Gajabhiye, Tejas Auti, Harshal Nikam, and Pravin Jadhav from College of Engineering, Pune won the Gold award of INR 5 Lakhs. They had worked on creating centrally suspended cage-less Differential for automotive sector.
Silver award winner Shruti Kharwandikar and Kruti Patel from MKSSS's Cummins College of Engineering for Women, Pune.
Silver award winner: The team of Ajay Kandgal and Vivekanand Kulkarni from B.V. Bhoomaraddi College of Engineering and Technology, Hubli
The team of Sourabh Alagundagi, Rajat Bapuri, Sumitra Malagi from B.V. Bhoomaraddi College of Engineering and Technology, Hubli
Follow us
Contact us
Get in touch with us at
KPITSparkle2022@kpit.com


It’s Not a CET PRACTICE , IT’S REAL APOLOGIZE LETTER.
Now,right after the first class began i just feel this course isn’t for me at all. Besides i am here to write this just to give you my sincerity regret to you,the girl i hurt last night without being conscious of it, the girl i met on the airplane a month ago.Because i do value the friendships we’ve built and i do want it can goes on and on and on…. First of all,please allow me to say it again “i am sorry”. I have said it for thousands times before,but still. Hopefully it will work this time after the second when you read it over .
I should NOT put any of my personal judgement on you when i didn’t fully think it over,especially to those messages that may hurt your feelings and its not a excuse to simply say i just didn’t realize what i typed may hurt feelings. I am deeply sorry about that. Then,i am kind of glad that you were mad at me,maybe you also value this friendship,too. Even though it’s not as much as i do. But i still grateful for that,not only because you pointed my shortcomings out but also your can totally spilled your anger on me,which kind of showed that you’re not longer considering me as a passenger or flymate :D Nevertheless,i still feel sorry for you and i am begging you for your forgiveness that from the deepest part of your heart. Please give me a second chance and i will do anything i can possibly do to fix the mistake i have ever made.
Least but not least,apart from my sincerity regret,do you remember to help me fetch my stuffs that almost arrive here. uhh,two classes almost come to an end but i still wanna to post something off the topic but it’s sort of related to our friendship. When i asked you to give me your number,you just gave me your qq with a poor excuse by saying your phone’s charge was overdue while your qq stayed logged in.
Now,I can finally get you number and i can have chances to dial it one day,may I? Okay , class is almost over but my regret always alive till you forgive me.
Yours sincerely.
written at 11:42am


Ring - Marla
Circular buffer.struct marla_Ring
| Member | Description |
|---|---|
| char* buf | Internal buffer. |
| unsigned int read_index | Offset of the read head in the internal buffer. Not guaranteed to be in [0, capacity). |
| unsigned int write_index | Offset of the write head in the internal buffer. Not guaranteed to be in [0, capacity). |
| size_t capacity | Capacity, in bytes, of the internal buffer. |
marla_Ring* marla_Ring_new(size_t capacity)
Creates and returns a new marla_Ring with the specified capacity. The capacity must be a power of two.void marla_Ring_free(ring)
Destroys and frees the given ring.unsigned int marla_Ring_size(ring)
Returns the number of bytes available for reading from this ring.size_t marla_Ring_capacity(ring)
Returns the capacity of the ring.int marla_Ring_readc(marla_Ring* ring, unsigned char* out)
Reads a single character from the ring. Returns 1 if a character was read, 0 otherwise.int marla_Ring_read(ring, char* sink, size_t size)
Reads up to the specified number of bytes from the ring. The number of bytes actually read is returned.int marla_Ring_isFull(marla_Ring* ring)
Returns 1 if the given ring is full of data to be read.int marla_Ring_isEmpty(marla_Ring* ring)
Returns 1 if the given ring is empty.void marla_Ring_putbackRead(ring, size_t count)
Reverses this ring's read head by the specified number of bytes.void marla_Ring_putbackWrite(ring, size_t count)
Reverses this ring's write head by the specified number of bytes.size_t marla_Ring_write(ring, const char* source, size_t size)
Writes up to the specified number of bytes from the given source to this ring. The number of bytes actually written is returned.int marla_Ring_writec(ring, char source)
Writes the specified character to this ring. Returns 1 if the character was written, or 0 if the ring is full.int marla_Ring_writeStr(ring, const char* source)
Shorthand formarla_Ring_write(ring, source, strlen(source))
.


The 30th International Conference on Antiviral Research was held in Atlanta, GA, USA, from 21 to 25 May 2017. Each year, the International Society for Antiviral Research (ISAR) presents three major awards, this year to Mike Sofia (Elion award), David Chu (Holý award) and Maaike Everts (Prusoff award). Also this year, the inaugural ISAR Women in Science award lecture was presented by Priscilla Yang. For several years, International Conference on Antiviral Research (ICAR) has included at least one Keynote lecture, this year there were four. Although there are accounts of only these eight lectures, they reflect the diversity that is characteristic of ICAR – employment (academia, industry, public health), type of research (virus biology, potential antiviral targets, antiviral drugs, research organisation) and a range of viruses. For example, the viruses included were hepatitis C virus and hepatitis B virus (Mike Sofia), HIV and hepatitis B virus (David Chu), multiple antiviral projects (Maaike Everts), dengue (Priscilla Yang), rhinovirus C (Ann Palmenberg), polio (Mark Pallansch), HIV (Eric Hunter) and Zika virus (Pei-Yong Shi). This report ends with my personal comments giving examples in which this diversity can bring benefits. The 31st ICAR will be in Porto, Portugal, 11–15 June 2018.


Backstabbr Options
Private mode
Hide tooltips
Screenshot
Pressing
space
enters
emergency private mode.


Top Best Buy Black Friday 2019 deals
by Sean PortnoyWe combed through Best Buy's Black Friday 2019 ad and found the best tech deals from its sale.
Best Buy's Black Friday ad is always one of the most anticipated of the holiday shopping season, and one pleasant surprise with its release this year is that many of the deals are already available, a sign that online shopping continues to erode the "magic" of Black Fridays past when people trampled each other to get to doorbuster specials before they ran out.
We've covered Best Buy's Black Friday Apple deals elsewhere, but below are several other sales that caught our eye.
When is Best Buy's Black Friday Sale?
- See it now: Best Buy's Black Friday 2019 sale
Best Buy has already announced all the Black Friday 2019 deals it plans to offer in its annual Black Friday ad Luckily for us, some of these deals are now live, but others won't be live until Nov. 28. Best Buy is also highlighting a new sale daily with its special Deal of the Day promotion.
Black Friday 2019: Top Best Buy deals
If these deals aren't yet available, they will be when Best Buy's sale goes live Nov. 28.
Ring bundle with Echo Show 5 for $180 ($210 off)
- See it now: Best Buy
This bundle comes with the Ring Video Doorbell Pro, Chime Pro, and Echo Show 5 -- making it an amazing smart home starter kit. The Ring Video Doorbell Pro usually goes for $190 alone.
Samsung 3 Chromebook 11.6-inch laptop for $90
- See it now: Best Buy
The lowest price we've seen for a new laptop so far this Black Friday comes courtesy of this Chromebook, which is $10 less than the HP Chromebook 11-v010nr deal at Target and elsewhere. On the other hand, it comes with half the RAM and built-in storage of the HP, and its Intel Atom x5 CPU is less powerful than the HP's Celeron processor. But if you want the absolute cheapest notebook, Samsung is the winner thus far.
Lenovo IdeaPad 1 14-inch laptop for $130
- See it now: Best Buy
For a little more money, the IdeaPad provides quite a bit more computer than the Samsung Chromebook deal does, while still costing less than $150. It's equipped with an AMD A6-9220E dual-core processor, 4 gigs of RAM, 64GB of storage, and a 14-inch display to run Windows 10 instead of Chrome OS.
Microsoft Surface Pro 7 12.3 inches for $600 ($150 off)
- See it now: Best Buy
Microsoft is currently selling its latest Surface Pro tablet for $749, so this is a nice chunk of change off that price. This base model comes with an Intel Core i3 chip, 4GB of RAM, 128GB SSD, and a 12.3-inch 2,736x1,824 touchscreen display.
HP Pavilion 23.8-inch All-in-One PC for $700
- See it now: Best Buy
A nicely designed all-in-one, this HP also has quality specs for the price. It comes with a Core i5-8400T six-core CPU, 12GB of memory, a 256GB SSD, and a 23.8-inch display with full HD resolution and touch capabilities.
CyberPowerPC Gamer Master Gaming Desktop for $600 ($150 off)
- See it now: Best Buy
While it's named Gamer Master, and it's designed like one, this is priced and spec'd for anyone looking for a powerful midrange desktop (and can handle the colorful touches). Built around an AMD Ryzen 5 3600 six-core processor, it also includes 8GB of RAM, both a 2TB hard drive and 240GB solid-state drive, and an AMD Radeon RX 580 graphics card.
Asus ROG Strix SCAR III G731GU Gaming Laptop for $900 ($330 off)
- See it now: Best Buy
Gamers looking for a full desktop replacement for under a grand should take a long look at this deal on an Asus Republic of Gamers (ROG) notebook. It's powerful -- Core i7-9750H processor, 16GB of RAM, 512GB SSD, and Nvidia GeForce GTX 1660 Ti -- if not highly portable with a 17.3-inch full HD screen and total weight a bit over 6 pounds. It's clearly designed for those who don't plan to move it around that much but still want the ability to take a rig with them.
More Best Buy Black Friday 2019 deals
Best Buy's Black Friday ad runs 119 pages long, so there are plenty of other deals beyond the half-dozen we highlighted above. Below are the best of the rest.
Laptops, tablets, and desktops
-
Dell Inspiron 15.6-inch Laptop (Core i5 processor, 8GB of RAM, 256GB SSD, touchscreen) for $349.99
- HP Pavilion x360 2-in-1 Laptop (Intel Pentium processor, 4GB of RAM, 128GB SSD, 11.6-inch touchscreen) for $279.99
- Google Pixel Slate Chrome OS tablet (Core m3 processor, 8GB of RAM, 64GB SSD, 12.3-inch touchscreen) for $449.99
- HP Omen Obelisk Gaming Desktop (Core i7-9700 processor, 16GB of RAM, 1TB hard drive, Nvidia GeForce GTX 1660 Ti graphics) for $899.99
- Dell 17.3-inch Gaming Laptop (Core i7 processor, 16GB of RAM, 1TB hard drive, Nvidia GeForce RTX 2060 graphics) for $1,099.99
Other tech
- Beats Studio3 Noise Canceling Headphones for $200 ($150 off)
- Dyson Cyclone V10 Motorhead Vacuum for $300 ($200 off)
- Amazon Fire TV Stick 4K for $25 ($25 off)
- Samsung Galaxy Watch for $150 ($50 off)
Disclosure: ZDNet may earn a commission from some of the products featured on this page.


&; OpenUrl leaf Search latitudinal site herbivory variation climatic attack insect bird oak gall-inducers model plant precipitation temperature CrossRef leaf-miners species University caterpillars Figure defences latitude rates effect incidence significant effects higher leaves Department Moreira gradients Ecology Ecology, mean traits chemical dummy herbivore interactions models represent scientists Forest Science Web increasing patterns Table chemistry data predation professional Research Roslin best climate soluble study trees used Latitudinal [[#xB]] herbivory, lower tree Anstett Biology France collected i.e., predictors rate studies Czech Global S). Sciences, bottom-up clines diversity geographic increased marks partners predators schoolchildren set spring sugar top-down C:N However, INRAE, Kozlov factors gradient herbivores herbivory. latitudes response variables Andreas Anna European Faculty Quercus Thomas University, associated content dashed different line plant-herbivore tropical variable Biogeography, Biology, Partner Republic Science, averaged biotic concentration conditions decreased evidence included relationships sampled trees. (e.g. (see .-%, Abdala-Roberts, Belgrade, Budejovice, Castagneyrol, Ceske Coley, Download E.L. Institute, Plant Rasmann, Solid Suceava, Valdés-Correcher Zvereva, corresponding did drivers geographical gradients. growing importance including left levels lines material new non-significant nutritional partner predictions present project provided range ratio relationship temperate underlying using (and Abstract BIOGECO, Bordeaux, Castagneyrol Centre Cestas, Chen, Elena Entomology, Europe Europe. Insect Kozlov, Leaf Moles Moreira, Open Price, Univ. Zverev, arthropod based birds calculated data-hide-link-title=""> data-icon-position="" defences, ecology figure identified induced insects interactions. large-scale low measured mechanisms mediated model, natural null parameter pedunculate plants precipitation, proportion raw respectively. schoolchildren. selection simultaneously species. strength tab tannins temperature, trends (Figure Academy Agrawal, Alexander Andrea Andrew Andrey Anne-Maïmiti Arndt Augusto Aurélien Ayco Barbaro Barone, Bastien Biodiversity Biological Birmensdorf, Bouget Bouriaud Branco Centenaro Christophe Coley Csaba Csóka Damestoy Daniela Deborah Dobrosavljević Dots Duduman Dulaurent Ecological Evolution, Eötvös Faticov Federal Ferrante Finland Forestry Forestry, Fürjes-Mikó Galmán Giada Gordon Gossner György Gábor Hampe Harvey Howe Institute J.D. J.L. Johnson, Jovan Juha-Matti Julia Kadiri Kaennel-Dobbertin Katerina Koricheva Kozel Lars Lassi Laurent Leaf-miners Letters, Luc Lupaştean Löveï M.T.J. Manuela Marco Maria Marija Marketa Martin Material Michael Michèle Mihai-Leonard Mikhail Milanović Model Mrazova N:P National Oikos, Olivier Opgennoorth Petersburg Pitkänen Popova Popović Prinzing Queloz Rebecca Regression Sallé Sam Schemske Scherer-Lorenzen Schuldt Selikhovkin Slade, Slobodan St. Suominen Supplementary Swiss Switzerland Tack Tahadlova Thus, Tomas Turku, Valentin WSL, Xoaquín Yasmine Zvereva Zürcherstrasse [[#x]]. abiotic adaptive analyses arthropods avian birch branches caterpillar complete condensed constitutive correlation defensive equations error error) explain figures follows: forces forests. given global herbivores. high hydrolysable important influenced intermediate intraguild lack latitudes, level. light local models. multiple non-linearly northern phenolics plant-herbivore-predator positive potential prey quantified rate, ratio, results robur sent shown. size standard str. supplementary survey symbols) trend types understanding value. x<;sup>;<;/sup>;; Ágnes ΔAICc