Event Location:
Northcentral Technical College
1000 W Campus Dr
Wausau, Wisconsin
Abstract:
This session is designed to be useful to two different audiences: 1.Those implementing Ola Hallengren’s Maintenance Solution for the first time. (Or don't even know what the Maintenance Solution is!) 2.Those that have been using it for awhile, but might have a hole in their DR plan – what to restore when a resource goes offline.  What I am presenting is a custom extension that works out of the box for jobs created using the maintenance solution. The scripts will add an additional job step to all DatabaseBackup jobs which will output a text file that contains the most current restore scripts for each database in the instance. If you don't know what I just described in that last paragraph, you should come to this session and that will be explained as well!
Speaker(s): Jared Zagelbaum,
Abstract:
SQL Server 2016 was released in June 2016 with tons of new features. When Service Pack 1 was released in November 2016, many of the Enterprise level features were allowed in lower editions of SQL server 2016. Microsoft made sure everyone got something out from an upgrade to SQL Server 2016. In this session, you will get an overview of several of the new features in SQL Server 2016. It will cover Dynamic Data masking, Row Level Security, Always Encrypted, Temporal tables and time permitting few more goodies. This is a high-level overview session on advanced features of SQL Server 2016 and therefore we will not be going deep into any one technology. My goal is to inspire you to take action and play with new features of this great product.
Speaker(s): Ameena Lalani,
Abstract:
You know that execution plans can provide you with vital information to tune a query, but just how does that plan get generated? In this session, we will examine details the query optimization process that are not very well known. We will discuss how SQL Server parses a T-SQL statement then and begins applying rules to make it more efficient. Did you know that SQL Server can be coaxed into outputting hidden structures such as parse trees and memos? We'll do just that, and then take it a step further with a unique visualization tool that I have created. With a better understanding of how SQL Server optimizes T-SQL (and sometimes doesn't do it as well as might be hoped), you can write more efficient SQL statements.
Speaker(s): Brian Hansen,
Abstract:
You’re busy. You don’t have time to check the security of your systems all the time, right? It’s not like anyone has added a user that you don’t know about. And no one would create a database and add all logins to the db_owner role. And of course, no one would ever think of setting the TRUSTWORTHY property on a database... right? Things like that happen on a daily basis, poking holes in even the most well-secured database. Those holes are just waiting for someone to find them. That someone could be you, or it could be an attacker probing for a way to get data from your company. In this session, we will explore how to create reports that are quick to review so you can find the holes before someone else does - without interfering with the rest of your day. Along the way, you'll also learn where to find security-related information in SQL Server's metadata and logs to allow you to customize and extend the reports to meet your company's unique needs.
Speaker(s): Ed Leighton-Dick,
Abstract:
Business Intelligence has sat pretty comfortably on the same laurels for well over a decade however that landscape has been disrupted by the surfacing new Azure tools, Power BI, and now Power BI on premise. With so many new tools and features, things can get pretty intimidating, especially for those with additional considerations and regulations. It can, however, open new doors to organizations who were previously unable to fully utilize BI. Through presentation, demo, and open discussion we attempt to navigate classic BI Architecture and the available options there in, as well as touch upon the future of BI and modern architecture.
Speaker(s): George Bryant,
Abstract:
SQL Server is optimized to work in sets, but this is certainly a paradigm shift for someone accustomed to row-by-row operations. We will look at T-SQL features that can break set-based processing -- generally meaning poor performance -- and gather empirical evidence to support that assertion. We'll also look at some constructs to overcome these issues and make your T-SQL scripts run faster and use fewer resources.
Speaker(s): Brian Hansen,
Abstract:
As DBAs, we are asked all sorts of impossible questions. Who dropped that important table last month? Why was everything slow last week? Who made that user a sysadmin? You’d have to be a mind reader to know the answers to some of those, right? Not necessarily! SQL Server can help you find information about these and other common problems with its default trace and system_health Extended Events session, but you have to know how to collect and interpret the data they provide. In this session, we’ll explore what system_health and the default trace are, what information they contain, and how to use them to find the answers you’re searching for.
Speaker(s): Ed Leighton-Dick,
Abstract:
In SQL Server, the transaction log is the most important file in any database. All changes to the data and schema are tracked in the transaction log. Join me to learn how to read the transaction log, how to follow a transaction as is processes, and how SQL Server handles a transaction rollback. You will also learn how to identify the resources and objects SQL Server accesses during the course of the transaction. Finally, you will see how SQL Server processes transactions for In-Memory tables.
Speaker(s): Frank Gill,
Abstract:
The SQL transaction log can be a mystery to many DBAs. We live in the age of visualizations, so let's visualize the log! In this session, we will examine the purpose and organization of the transaction log and will explore the log through a unique visualization tool. We will consider common performance issues that are related to the logging mechanism and how to overcome these problems. Finally, we will discuss some techniques to monitor the log so that potential performance problems can be addressed pro-actively.
Speaker(s): Brian Hansen,
Abstract:
Everybody loves a free lunch. Come to this session to learn about all the SQL Server freebees.
Speaker(s): Cecil Spivey,
Abstract:
SQL Server 2016 came out with a bucket full of cool new features. One of them is Temporal Tables. They are basically history tables of the normal tables; that allow us to retrieve data from any point in time for every change such as updates and deletes. It does this all without any change to application code and without using any behind the scene triggers. In this session, we will see what the requirements of creating temporal tables are. We will see the demo on insert and update functionality and how to query historical information using new TSQL clause. It has lots of demos and less theory.
Speaker(s): Ameena Lalani,
Abstract:
Microsoft has recently streamlined the certification process and this session focuses on what you need to know to get Microsoft Certified on SQL Server. The session is led by an expert Microsoft Certified Trainer (MCT) and Microsoft MVP who regularly teaches the SQL Server certification courses. The presenter walks you through the certifications, exams, and resources available to get you ready for success.
Speaker(s): John Deardurff,
Abstract:
This session with provide a high level overview of what SQL Server Analysis Services is and what you need to know to get started. It will cover the fundamentals of Dimensions, Measures, how to view the data as well as the role of SSAS in the enterprise. The sessions with also provide a few scenarios demonstrating the benefits of using SSAS.
 Curious about what's inside, or how they are built? Follow along in this session as we build a data warehouse from scratch. Learn about fact tables, dimension tables, measures, and attributes as we work through a small data warehouse using Microsoft's newest sample database. In this beginner level session, we will talk about data warehouse concepts, build a small data warehouse with a fact table and several dimensions, and implement an ETL process using T-SQL and stored procedures.
Speaker(s): Jesse Seymour,
Abstract:
SQL Server tracks all changes to a database's data and schema in the transaction log. This makes it the most important file in any database. Understanding transaction log architecture and behavior will help you develop code that will maximize performance and minimize resource consumption. Join me to learn about transaction log architecture , batching transactions, rollback activity, and the effect these things can have on performance.
Speaker(s): Frank Gill,
Abstract:
This is a introductory session on how a DBA and developers can use Execution plans to find performance bottlenecks. This will cover the fundamentals and cover some of the basic techniques that can be used to find hidden performance problems.
Speaker(s): Dave Bland,
Abstract:
In this session of Getting Started with Azure Data Lake Analytics, we will learn the fundamentals of designing a Data Lake, how that fits into your organization, and then spend most of the time processing data with Data Lake Analytics in U-SQL.
Speaker(s): Sean Forgatch,
Abstract:
Azure SQL Database provides an environment to quickly build SQL Server databases. This allows for testing of new versions and functionality with minimal effort. PowerShell provides cmdlets that create, configure and deploy an Azure SQL Database quickly and easily. Additionally, cmdlets exist to tear the environment down once you are done with it, Join me to learn more about the options available for scripting and automating the build of an Azure SQL Database.
Speaker(s): Frank Gill,
Abstract:
Come chat with our panel of SQL experts for an hour long QA session. From DBA 101 to advanced internals, any question is fair game!
Speaker(s): Krista Olson,
Abstract:
Per Microsoft, SQL Server 2016 “is the biggest leap forward in Microsoft's data platform history with real-time operational analytics, rich visualizations on mobile devices, built-in advanced analytics, new advanced security technology, and new hybrid cloud scenarios.” There’s a ton of ground to cover on all of the new enhancements, so this session will focus on some of the new advanced security offerings that Microsoft has incorporated into the platform: Dynamic Data Masking, Row-Level Security, and Always Encrypted.
Speaker(s): Jared Zagelbaum,
Abstract:
If you have used a database, chances are almost certain you've utilized indexes as well. In this presentation I will discuss both Rowstore and Columnstore Indexes and why they are important to anyone that interacts with a database. This session will cover what they are, how they are utilized, how best to take advantage of them, and even when they can be problematic. It is my intention to help anyone become more comfortable with indexes and understand what they can do for you and your role, be it a DBA, Developer, or BI Professional.
Speaker(s): John Eisbrener,
Abstract:
Since Extended Events are replacing Profiler, every DBA should know what they are and how to use them. This session will cover the basics of Extended events as well as an introduction to the system_health session. This session will give someone who has never used Extended events before the skills to start using them!
Speaker(s): Dave Bland,
Abstract:
In this session we'll develop a Tabular Data Model in MS SSAS and review our options for deploying, sharing and visualizing it. Well also discuss how senior leaders will see the business value of SSAS and how end users can utilize it for Self-Service Business Intelligence.
Speaker(s): Wylie Blanchard,
Abstract:
Determine which model presents the right business value for your company's analytical needs. In this session we'll discuss the difference between SSAS Tabular and SSAS Multidimensional modes, feature differences as well as the benefits and cons to utilizing both modes in MS SQL Server 2016.
Speaker(s): Wylie Blanchard,
Abstract:
This session will cover what is needed to transition from someone who knows SQL Server to becoming a great DBA. It will cover what technical skills are needed, how to think like a great DBA and explore some of the functionality of SQL Server that can help all DBAs do their job. The presenter will share of the his experiences of his journey to go from a trainer to a successful DBA.
Speaker(s): Dave Bland,
Abstract:
We’ve all seen the recent news stories about companies whose data has been stolen by hackers. What was once a rare event has become all too common, and companies large and small are at risk. While it isn’t always possible to prevent intrusions, you can reduce the risk by encrypting your data. In this presentation, I’ll show you the four ways that SQL Server provides to encrypt data: hashes, cell-level encryption, database-level encryption (also known as transparent data encryption), and backup encryption. We’ll also discuss the keys required for each type of encryption and discuss how to protect the keys themselves.
Speaker(s): Ed Leighton-Dick,
Abstract:
There is a vast array of material in the SQL Server community that covers the in-memory engine in great detail, this session will cover some of the internals of the in-memory engine, however what it will also do is fill in the void as to common performance issues and how they can be solved with the in-memory engine, specifically: - Staging data as fast as possible - Turbo charging scalar aggregate functions - Implementing a scalable sequence - Overcoming the performance limitations of temporary tables - Making the critical path in your code base run faster
Speaker(s): Christopher Adkin,
Abstract:
In SQL Server 2016 Microsoft has include native handling of JSON. What does that mean? What is JSON? Why do I care? Will there be knives and hockey masks? Come to this session and you will learn what JSON is, where and how it is used and why as a data professional you need to know this.
Speaker(s): Jim Dorame,
Abstract:
Always Encrypted is a highly-touted new feature of SQL Server 2016 that promises to make encryption simple to use and transparent to applications while still protecting the data both at rest and in motion, even from high-privilege users such as developers and DBAs. Does that sound too good to be true? It isn’t - Always Encrypted is an incredible feature - but like any new technology, it does have some limitations. In this session, you’ll see how to configure Always Encrypted, and we’ll talk about when you should and shouldn’t use it in your environment.
Speaker(s): Ed Leighton-Dick,
Abstract:
This session is for the absolute beginner to learn the terminology and the PROCESS of queries. This includes how queries are Parsed, Resolved, Optimized, Compiled and Executed. We will see how this effects Batches, Transactions, and Errors. We will also briefly discuss table structures and the differences between Heaps and Clustered Indexes. The session is led by an expert Microsoft Certified Trainer (MCT) and Microsoft MVP who regularly teaches the SQL Server certification courses.
Speaker(s): John Deardurff,
Abstract:
Come to this session to get your toes wet with machine learning. What types of questions it can answer? What types of models it can produce? What are some techniques used? There will be practical demos using Azure Machine Learning as well.
Speaker(s): Jared Zagelbaum,
This is a list of speakers from the XML Guidebook records. The details and URLs were valid at the time of the event.
LinkedIn: http://http://www.linkedin.com/profile/view?id=22766305amp;trk=tab_pro
Contact: http://skreebydba.com/
Frank Gill is a Senior Data Engineer at Concurrency With 21 years of IT experience, the first 8 as a mainframe programmer, he has developed a love of all things internal. Over the past several years, he has worked extensively with SQL Server solution in Azure, including Managed Instance. When not administering databases or geeking out on internals, Frank volunteers at the Art Institute of Chicago and reads voraciously.
LinkedIn: https://www.linkedin.com/in/dave-bland-sql-server
Contact: https://www.davebland.com
Over 20 years of SQL Server experience that includes being a DBA, BI development and Application development using VB.NET. Dave currently is the Manager of the DBA team at Stericycle. Dave is a Friend of Red-Gate for 2019. He is a frequent presenter at SQL Saturday events and user groups around the country. Has been teaching SQL Server since version 2000 and is the SQL Server instructor at Harper College in Palatine, IL. Current certifications include: • Microsoft Certified Trainer • MCTS: SQL Server 2008, Implementation and Maintenance • MCTS: SQL Server 2008, BI Development and Maintenance • MCSA: SQL Server 2008 • MCSA: SQL Server 2016 • MCSE: SQL Server 2016 - BI • MCDBA: SQL Server 2000 • MCSD • MCSE: Data Management and Analytics
LinkedIn: https://www.linkedin.com/in/mrgbryant
George has over a decade of experience in the Business Intelligence industry and is an active member of the BI community across the Midwest. His knowledge spans multiple industries including, retail, manufacturing, finance, and medical, however recent years has seen him focus on the analytics in sports. George specializes in BI architecture and end to end development in order to deliver advanced analytics and metrics to multiple organizations across the country. In recent years George has focused on and worked primarily in the Azure cloud, utilizing both established and recent offerings.
LinkedIn: https://www.linkedin.com/in/dave-bland-sql-server
Contact: https://www.davebland.com
Over 20 years of SQL Server experience that includes being a DBA, BI development and Application development using VB.NET. Dave currently is the Manager of the DBA team at Stericycle. Dave is a Friend of Red-Gate for 2019. He is a frequent presenter at SQL Saturday events and user groups around the country. Has been teaching SQL Server since version 2000 and is the SQL Server instructor at Harper College in Palatine, IL. Current certifications include: • Microsoft Certified Trainer • MCTS: SQL Server 2008, Implementation and Maintenance • MCTS: SQL Server 2008, BI Development and Maintenance • MCSA: SQL Server 2008 • MCSA: SQL Server 2016 • MCSE: SQL Server 2016 - BI • MCDBA: SQL Server 2000 • MCSD • MCSE: Data Management and Analytics
LinkedIn: http://http://www.linkedin.com/profile/view?id=22766305amp;trk=tab_pro
Contact: http://skreebydba.com/
Frank Gill is a Senior Data Engineer at Concurrency With 21 years of IT experience, the first 8 as a mainframe programmer, he has developed a love of all things internal. Over the past several years, he has worked extensively with SQL Server solution in Azure, including Managed Instance. When not administering databases or geeking out on internals, Frank volunteers at the Art Institute of Chicago and reads voraciously.
LinkedIn: https://www.linkedin.com/in/dave-bland-sql-server
Contact: https://www.davebland.com
Over 20 years of SQL Server experience that includes being a DBA, BI development and Application development using VB.NET. Dave currently is the Manager of the DBA team at Stericycle. Dave is a Friend of Red-Gate for 2019. He is a frequent presenter at SQL Saturday events and user groups around the country. Has been teaching SQL Server since version 2000 and is the SQL Server instructor at Harper College in Palatine, IL. Current certifications include: • Microsoft Certified Trainer • MCTS: SQL Server 2008, Implementation and Maintenance • MCTS: SQL Server 2008, BI Development and Maintenance • MCSA: SQL Server 2008 • MCSA: SQL Server 2016 • MCSE: SQL Server 2016 - BI • MCDBA: SQL Server 2000 • MCSD • MCSE: Data Management and Analytics
LinkedIn: https://www.linkedin.com/in/jesseseymour
Contact: http://www.jesseseymour.com
Jesse Seymour is a business intelligence hack of all trades. From data warehousing to report writing, Jesse Seymour is responsible for the complete business intelligence environment of the Forest County Potawatomi. In his free time, Jesse is one of the chapter leaders of WausauPASS, Wisconsin's newest PASS chapter. Involved in IT since the age of 16, Jesse Seymour has played nearly the entire field of information technology, with over three years experience in business intelligence and over five years experience in Microsoft SQL Server.
LinkedIn: http://www.linkedin.com/in/JohnDeardurff
Contact: https://www.sqlmct.com
John has been a Microsoft Certified Trainer (MCT) for over 20 years teaching Azure, SQL Server, Exchange Server, and Windows Server networking courses. He is currently a SQL Premier Field Engineer for Microsoft. He is an MCT Regional Lead for the Eastern United States and a former Data Platform MVP.
LinkedIn: http://www.linkedin.com/pub/john-eisbrener/4/356/b36/
Contact: http://www.dbatlas.com
John is a DBA (Default Blame Assignee) with over a decade of experience breaking and subsequently fixing, database solutions. He maintains separate swear jars for Oracle, Postgres, and Greenplum, but keeps the largest container reserved for SQL Server. When not accumulating loose change for his children's college funds, John can be found trying to help others with their database solutions as the Principal Consultant for DB Atlas, speaking at various SQL Saturdays throughout the Midwest, or helping out with his local PASS chapter meetings in Madison, WI.
LinkedIn: http://www.linkedin.com/in/wylieblanchard
Contact: https://WylieBlanchard.com/?utm_source=sqlsaturday.comutm_medium=profileutm_campaign=website
Wylie Blanchard, MCSE is an IT consultant, SQL Server professional and web strategist who enjoys helping people learn how to use technology as a tool to amplify their professional lives. He has 15+ years of experience in the IT Industry and holds several certifications in SQL and Oracle systems. He is a father of twin-girls, a gadget enthusiast and a self-proclaimed pizza connoisseur. Learn more about Wylie at: www.WylieBlanchard.com.
LinkedIn: https://www.linkedin.com/in/jaredzagelbaum?
Contact: https://jaredzagelbaum.wordpress.com
With over 10 years of experience using the Microsoft Data Platform suite, Jared’s main areas of focus include data lake architecture, machine learning, and application embedded analytics. Jared regularly speaks at user groups and events across the Midwest, as well as blogs and provides technical review for SQL Server publications.
LinkedIn: http://www.linkedin.com/in/eleightondick
Contact: http://www.edleightondick.com
Ed Leighton-Dick is a Microsoft MVP, SQL Server performance and architecture specialist, and Founder/Principal Consultant of Kingfisher Technologies. He is a frequent volunteer with PASS, including current roles as a Regional Mentor, co-leader of I-380 PASS local group, and co-leader of the HA/DR virtual group. He can often be found teaching sessions at local, regional, and national events, including user groups, SQLSaturday, and PASS Summit.
LinkedIn: http://www.linkedin.com/in/JohnDeardurff
Contact: https://www.sqlmct.com
John has been a Microsoft Certified Trainer (MCT) for over 20 years teaching Azure, SQL Server, Exchange Server, and Windows Server networking courses. He is currently a SQL Premier Field Engineer for Microsoft. He is an MCT Regional Lead for the Eastern United States and a former Data Platform MVP.
LinkedIn: https://www.linkedin.com/in/ameena
Ameena Lalani is a SQL Server veteran and started her journey with SQL Server 2000. She is a Microsoft Certified Solution Associate on SQL Server 2016 and also hold Azure Administrator Associate certification. Ameena works at Microsoft as a Premier Field engineer. She has implemented numerous High Availability and Disaster Recovery solutions at various companies. Ameena loves to share her technical knowledge and speaks at local user groups and SQL Saturday events throughout the United States.
LinkedIn: https://www.linkedin.com/in/wollatondba
Contact: http://www.chrisadkin.io
Chris is a freelance SQL consultant who has been working with SQL Server since 2000, his passion is for squeezing every last drop of performance out of SQL Server and understanding the database engine at a level which conventional tools cannot provide any insights on.
LinkedIn: https://www.linkedin.com/in/jamesdorame
Contact: http://www.jamesdorame.com/
Jim Dorame has been working with SQL Server for over 20 years. Over those years he has learned a great deal from the SQL community. He has also given back to the community in the hopes that he may help someone in their career. He is always looking for something to learn and luckily Microsoft keeps adding to SQL Server.
LinkedIn: https://www.linkedin.com/in/dave-bland-sql-server
Contact: https://www.davebland.com
Over 20 years of SQL Server experience that includes being a DBA, BI development and Application development using VB.NET. Dave currently is the Manager of the DBA team at Stericycle. Dave is a Friend of Red-Gate for 2019. He is a frequent presenter at SQL Saturday events and user groups around the country. Has been teaching SQL Server since version 2000 and is the SQL Server instructor at Harper College in Palatine, IL. Current certifications include: • Microsoft Certified Trainer • MCTS: SQL Server 2008, Implementation and Maintenance • MCTS: SQL Server 2008, BI Development and Maintenance • MCSA: SQL Server 2008 • MCSA: SQL Server 2016 • MCSE: SQL Server 2016 - BI • MCDBA: SQL Server 2000 • MCSD • MCSE: Data Management and Analytics
LinkedIn: http://www.linkedin.com/in/wylieblanchard
Contact: https://WylieBlanchard.com/?utm_source=sqlsaturday.comutm_medium=profileutm_campaign=website
Wylie Blanchard, MCSE is an IT consultant, SQL Server professional and web strategist who enjoys helping people learn how to use technology as a tool to amplify their professional lives. He has 15+ years of experience in the IT Industry and holds several certifications in SQL and Oracle systems. He is a father of twin-girls, a gadget enthusiast and a self-proclaimed pizza connoisseur. Learn more about Wylie at: www.WylieBlanchard.com.
LinkedIn: https://www.linkedin.com/in/jaredzagelbaum?
Contact: https://jaredzagelbaum.wordpress.com
With over 10 years of experience using the Microsoft Data Platform suite, Jared’s main areas of focus include data lake architecture, machine learning, and application embedded analytics. Jared regularly speaks at user groups and events across the Midwest, as well as blogs and provides technical review for SQL Server publications.
LinkedIn: http://www.linkedin.com/in/eleightondick
Contact: http://www.edleightondick.com
Ed Leighton-Dick is a Microsoft MVP, SQL Server performance and architecture specialist, and Founder/Principal Consultant of Kingfisher Technologies. He is a frequent volunteer with PASS, including current roles as a Regional Mentor, co-leader of I-380 PASS local group, and co-leader of the HA/DR virtual group. He can often be found teaching sessions at local, regional, and national events, including user groups, SQLSaturday, and PASS Summit.
Contact: http://www.tf3604.com
Brian is a database administrator at Children International in Kansas City. He has been working with SQL Server technologies since 1998, including roles in report development, application development and database administration.
LinkedIn: http://http://www.linkedin.com/profile/view?id=22766305amp;trk=tab_pro
Contact: http://skreebydba.com/
Frank Gill is a Senior Data Engineer at Concurrency With 21 years of IT experience, the first 8 as a mainframe programmer, he has developed a love of all things internal. Over the past several years, he has worked extensively with SQL Server solution in Azure, including Managed Instance. When not administering databases or geeking out on internals, Frank volunteers at the Art Institute of Chicago and reads voraciously.
Cecil is a DBA Engineer in the Twin Cities. He has been working in IT for over 20 years and was a consultant and a programmer/developer for many years until he was converted to the dark side.
LinkedIn: http://www.linkedin.com/in/eleightondick
Contact: http://www.edleightondick.com
Ed Leighton-Dick is a Microsoft MVP, SQL Server performance and architecture specialist, and Founder/Principal Consultant of Kingfisher Technologies. He is a frequent volunteer with PASS, including current roles as a Regional Mentor, co-leader of I-380 PASS local group, and co-leader of the HA/DR virtual group. He can often be found teaching sessions at local, regional, and national events, including user groups, SQLSaturday, and PASS Summit.
LinkedIn: https://www.linkedin.com/in/ameena
Ameena Lalani is a SQL Server veteran and started her journey with SQL Server 2000. She is a Microsoft Certified Solution Associate on SQL Server 2016 and also hold Azure Administrator Associate certification. Ameena works at Microsoft as a Premier Field engineer. She has implemented numerous High Availability and Disaster Recovery solutions at various companies. Ameena loves to share her technical knowledge and speaks at local user groups and SQL Saturday events throughout the United States.
Krista has been a DBA for over 20 years starting with SQL Server 6.5 and all versions since. She has worked with databases up to TBS in size, and managed over 450 servers with her creative and over technical counterparts on her team. She has her MCP in database administration. When not being a DBA you can find her on or in the water as a Wisconsin Lake Leader and president of McDill Lake District.
Contact: http://www.tf3604.com
Brian is a database administrator at Children International in Kansas City. He has been working with SQL Server technologies since 1998, including roles in report development, application development and database administration.
LinkedIn: https://www.linkedin.com/in/seanforgatch/
Contact: https://www.modern-dataengineering.com
Sean Forgatch is a Manager in Data Engineering consulting Avanade\Accenture based out of Chicago, IL focusing on Data Strategy, EIM , and Data Archtiecture. Serving multiple Fortune and Global 500 companies he has helped push the limits of Azure and is working on some of the largest Azure Data Lake and Analytics implementations in the industry. He has spoken at over 25 conferences across the United States since 2016, was the President of the Northeast Wisconsin PASS, and served on the board for SQL Saturday Madison.
LinkedIn: https://www.linkedin.com/in/jaredzagelbaum?
Contact: https://jaredzagelbaum.wordpress.com
With over 10 years of experience using the Microsoft Data Platform suite, Jared’s main areas of focus include data lake architecture, machine learning, and application embedded analytics. Jared regularly speaks at user groups and events across the Midwest, as well as blogs and provides technical review for SQL Server publications.
LinkedIn: http://www.linkedin.com/in/eleightondick
Contact: http://www.edleightondick.com
Ed Leighton-Dick is a Microsoft MVP, SQL Server performance and architecture specialist, and Founder/Principal Consultant of Kingfisher Technologies. He is a frequent volunteer with PASS, including current roles as a Regional Mentor, co-leader of I-380 PASS local group, and co-leader of the HA/DR virtual group. He can often be found teaching sessions at local, regional, and national events, including user groups, SQLSaturday, and PASS Summit.
Contact: http://www.tf3604.com
Brian is a database administrator at Children International in Kansas City. He has been working with SQL Server technologies since 1998, including roles in report development, application development and database administration.
The following is a list of sponsors that helped fund the event.


This document is the unified glossary used by all Open Credentials work in the Credentials Community Group.
This document is a work in progress and is being released early and often using an agile process; it is incomplete.
The group continues to actively align the Open Credentials terminology with definitions produced by other standardization bodies around the world.
This document is the unified glossary used by all Open Credentials work in the Credentials Community Group.
The cross reference section is highly experimental and may change drastically or disappear entirely over the course of the next several weeks or months.
This section contains cross references from terminology defined in this document to industry terminology defined in other specifications across all credentials-related standards groups.
The editors wish to thank the participants of the Credentials Community Group for discussions about and contributions to this document.


Rock your socks off with Crow Dawn in Uffculme
On Saturday, November 30, they will be at The Ostler Inn
by Lewis ClarkeCrow Dawn are making a welcome return to Uffculme next week.
On Saturday, November 30, they will be at The Ostler Inn Playing rock covers from the 60s to the present day.
Jim Clancy from the pub said: “Expect to be rocking your socks off to Led Zeppelin, Queen, Deep Purple, Status Quo, The Police, Pink Floyd, Beatles, AC/DC, Muse, Squeeze, Rush and many more.
“Mike, Rob, Tim and Steve are real entertainers who like to break the usual boundaries between the audience and band – and boy can they play.
“As usual the music starts at 9:00pm and it’s live and free.”


| < Day Day Up > |
3.3 Pretuning Your KernelSquid requires a fair amount of kernel resources under moderate and high loads. In particular, you may need to configure your system with a higher-than-normal number of file descriptors and mbuf clusters. The file-descriptor limit can be especially annoying. You'd be better off to increase the limit before compiling Squid. At this point, you might be tempted to get the precompiled binaries to avoid the hassle of building a new kernel.[1] Unfortunately, you need to make a new kernel, regardless. Squid and the kernel exchange information through data structures that must not exceed the set file-descriptor limits. Squid checks these limits at runtime and uses the safest (smallest) value. Thus, even if a precompiled binary has higher file descriptors than the kernel, the kernel value takes precedence.
To change some settings, you must build and install a new kernel. This procedure varies among different operating systems. Consult Unix System Administration Handbook (Prentice Hall) or your operating-system documentation if necessary. If you're using Linux, you probably don't need to recompile your kernel. 3.3.1 File DescriptorsFile descriptors are simply integers that identify each file and socket that a process has opened. The first opened file is 0, the second is 1, and so on. Unix operating systems usually impose a limit on the number of file descriptors that each process can open. Furthermore, Unix also normally has a systemwide limit. Because of the way Squid works, the file-descriptor limits may adversely affect performance. When Squid uses up all the available file descriptors, it is unable to accept new connections from users. In other words, running out of file descriptors causes denial of service. Squid can't accept new requests until some of the current requests complete, and the corresponding files and sockets are closed. Squid issues a warning when it detects a file-descriptor shortage. You can save yourself some trouble by making sure the file descriptor limits are appropriate before running ./configure. In most cases, 1024 file descriptors will be sufficient. Very busy caches may require 4096 or more. When configuring file descriptor limits, I recommend setting the systemwide limit to twice the per-process limit. You can usually discover your system's file-descriptor limit from your Unix shell. All C shells and similar have the built-in limit command. Newer Bourne shells and similar have a command called ulimit. To find your file-descriptor limits, try running these commands: csh% limit descriptors unlimited csh% limit descriptors descriptors 4096 or: sh$ ulimit -n unlimited sh$ ulimit -n 4096 On FreeBSD, you can also use the sysctl command: % sysctl -a | grep maxfiles kern.maxfiles: 8192 kern.maxfilesperproc: 4096 If you can't figure out the file-descriptor limit, Squid's ./configure script can do it for you. When you run ./configure, as described in Section 3.4, watch for output like this near the end: checking Maximum number of file descriptors we can open... 4096 If either limit, ulimit, or ./configure report a value less than 1024, you should invest the time to increase the limit before compiling Squid. Otherwise, Squid's performance will be poor under a moderate load. Increasing the file descriptor limit varies from system to system. The following sections offer some tips to help get you started. <IP_ADDRESS> FreeBSD, NetBSD, OpenBSDEdit your kernel configuration file, and add a line like this: options MAXFILES=8192 On OpenBSD, use option instead of options. Then, configure, compile, and install the new kernel. Reboot your system so the change takes effect. <IP_ADDRESS> LinuxConfiguring file descriptors on Linux is a little complicated. You must edit one of the system include files, and execute some shell commands before compiling and running Squid. Start off by editing the file /usr/include/bits/types.h. Change the value for _ _FD_SETSIZE as follows: #define _ _FD_SETSIZE 8192 Next, increase the kernel file descriptor limit with this command: # echo 8192 > /proc/sys/fs/file-max Finally, increase the process file-descriptor limit in the same shell in which you will configure and compile Squid: sh# ulimit -Hn 8192 This command must be executed as root and only works from the bash shell. There is no need to reboot on Linux.
<IP_ADDRESS> SolarisAdd this line to your /etc/system file: set rlim_fd_max = 4096 Then, reboot the system for the change to take effect. 3.3.2 Mbuf ClustersThe BSD-based networking code uses a data structure known as an mbuf (see W.R.Stevens' book, TCP/IP Illustrated, Vol 2). Mbufs are typically small (e.g., 128 octets) chunks of memory. The data for larger network packets are stored in mbuf clusters. The kernel may enforce an upper limit on the total number of mbuf clusters available in the system. You can find this limit with the netstat command: % netstat -m 196/6368/32768 mbufs in use (current/peak/max): 146 mbufs allocated to data 50 mbufs allocated to packet headers 103/6182/8192 mbuf clusters in use (current/peak/max) 13956 Kbytes allocated to network (56% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines In this example, there are 8,192 mbuf clusters available, but there are never more than 6,182 used at once. When the system runs out of mbuf clusters, I/O routines such as read( ) and write( ) return the "No buffer space available" error message. NetBSD and OpenBSD don't display mbuf usage in netstat -m output. Instead, they report "WARNING: mclpool limit reached" via syslog. To increase the number of mbuf clusters, you need to add an option to your kernel configuration file: options NMBCLUSTERS=16384 3.3.3 Ephemeral Port RangeEphemeral ports are the local port numbers the TCP/IP stack assigns to outgoing connections. In other words, when Squid makes a connection to an origin server, the kernel assigns a port number to the local socket. These local port numbers fall within a certain range. On FreeBSD, for example, the default ephemeral port range is 1024-5000. A shortage of ephemeral ports may adversely affect performance for very busy proxies (i.e., hundreds of requests per second). This is because some TCP connections enter a TIME_WAIT state when they are closed. An ephemeral port number can't be reused while the connection is in the TIME_WAIT state. You can see how many connections are in this state with the netstat command: % netstat -n | grep TIME_WAIT Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 <IP_ADDRESS>.19583 <IP_ADDRESS>.80 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.19597 <IP_ADDRESS>.80 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.19600 <IP_ADDRESS>.80 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.19601 <IP_ADDRESS>.80 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.19602 <IP_ADDRESS>.80 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.3128 <IP_ADDRESS>.25666 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.3128 <IP_ADDRESS>.25795 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.3128 <IP_ADDRESS>.1488 TIME_WAIT tcp4 0 0 <IP_ADDRESS>.3128 <IP_ADDRESS>.2194 TIME_WAIT Note that this example has both client- and server-side connections. Client-side connections have 3128 as the local port number; server-side connections have 80 as the remote (foreign) port number. The ephemeral port numbers appear under the Local Address heading. In this example, they are in the 19,000s. Unless you see thousands of ephemeral ports in the TIME_WAIT state, you probably don't need to increase the range. On FreeBSD, you can increase the range with this command: # sysctl -w net.inet.ip.portrange.last=30000 On OpenBSD, the command is almost the same, but the sysctl variable has a different name: # sysctl -w net.inet.ip.portlast=49151 On NetBSD, things work a little differently. The default range is 49,152-65,535. To increase the range, change the lower limit: # sysctl -w net.inet.ip.anonportmin=10000 On Linux, simply write a pair of numbers to the following special file: # echo "1024 40000" > /proc/sys/net/ipv4/ip_local_port_range Don't forget to add these commands to your system startup scripts so that they take effect each time your machine reboots. |
| < Day Day Up > |


from trello import TrelloClient
from pathlib import Path
from polical import configuration
import os
import yaml
import logging
[docs]def get_trello_client(user_dict: dict) -> TrelloClient:
"""This function gets a Trello Client Object to connect to Trello.
Args:
user_dict (dict): File path for the config file to be loaded
Returns:
client (TrelloClient): Client object that has access to Trello.
"""
client = TrelloClient(
api_key=user_dict["api_key"],
api_secret=user_dict["api_secret"],
token=user_dict["oauth_token"],
token_secret=user_dict["oauth_token_secret"],
)
return client
[docs]def get_uncompleted_cards() -> list:
"""This function gets a all uncompleted cards from a board.
Returns:
uncompleted_cards (List): List of strings from all uncompleted cards
"""
user = configuration.load_config_file("trelloCredentials.yaml")
client = get_trello_client(user)
all_boards = client.list_boards()
# last_board = all_boards[-1]
uncompleted_cards = []
for last_board in all_boards:
if last_board.name == "TareasPoli":
for list_from_board in last_board.list_lists():
for card in list_from_board.list_cards():
if card.due is not None and not card.is_due_complete:
uncompleted_cards.append(
card.name + " | " + list_from_board.name + "\n"
)
return uncompleted_cards
[docs]def get_cards_urls() -> list:
"""This function gets a all uncompleted cards from a board.
Returns:
uncompleted_cards (List): List of strings from all uncompleted cards
"""
user = configuration.load_config_file("trelloCredentials.yaml")
client = get_trello_client(user)
all_boards = client.list_boards()
last_board = all_boards[-1]
uncompleted_cards = []
last_list_from_board = last_board.list_lists()[-1]
for card in last_list_from_board.list_cards():
uncompleted_cards.append(
card.name + " | " + last_list_from_board.name + " | " + card.url + "\n"
)
return uncompleted_cards
[docs]def get_todo_tasks() -> list:
"""This function gets a all tasks that exits on todo.txt.
Returns:
tasks_list (List): List of strings from all tasks that exits on todo.txt
"""
with open(configuration.get_file_location("todo.txt"), "r") as todo:
tasks_list = todo.readlines()
return tasks_list
[docs]def write_todo_tasks(tasks_list: list):
"""This function writes all tasks from a list.
Args:
tasks_list (list): List containing tasks to be added to the todo.txt file
"""
with open(configuration.get_file_location("todo.txt"), "w") as todo:
todo.writelines(tasks_list)
[docs]def update_new_taks():
"""This function collects all new tasks from trello and concatenates to existing todo.txt file"""
current_tasks = []
current_tasks = get_todo_tasks()
uncompleted_tasks = []
uncompleted_tasks = get_uncompleted_cards()
for task in current_tasks:
if task in uncompleted_tasks:
uncompleted_tasks.remove(task)
if uncompleted_tasks:
current_tasks += uncompleted_tasks
write_todo_tasks(current_tasks)
else:
print("No existen nuevas tareas para añadir a todo.txt")
[docs]def get_done_tasks() -> list:
"""This function gets a all tasks that exits on todo.txt.
Returns:
tasks_list (List): List of strings from all tasks that exits on todo.txt
"""
with open(configuration.get_file_location("done.txt"), "r") as done:
readed_lines = done.readlines()
done_list = []
for done_item in readed_lines:
done_list.append(done_item.split(" ", 2))
return done_list
[docs]def update_done_taks():
"""This functions gets new done tasks and updates to the cards on trello"""
done_tasks_to_trello = []
for done in get_done_tasks():
splitted = done[2].split("|")
if len(splitted) == 2:
splitted[0] = splitted[0].rstrip()
splitted[1] = splitted[1].strip().replace("\n", "")
done_tasks_to_trello.append(splitted)
user = configuration.load_config_file("trelloCredentials.yaml")
client = get_trello_client(user)
all_boards = client.list_boards()
# last_board = all_boards[-1] # Error cuando se cree un nuevo board
for last_board in all_boards:
if last_board.name == "TareasPoli":
for done_task in done_tasks_to_trello:
for list_from_board in last_board.list_lists():
if list_from_board.name == done_task[1]:
for card in list_from_board.list_cards():
if card.name == done_task[0] and not card.is_due_complete:
card.set_due_complete()
print("Updated", card.name, "to DONE")
def generate_todo_txt():
print("Actualizando trello desde done.txt...")
update_done_taks()
print("Generando nuevas tareas para todo.txt...")
update_new_taks()


Class Times:
View upcoming classes. Free for 2021!
In your local time!
To join any or all classes Contact Jules via WhatsApp for the Zoom password.
About Us
What is
Aikido
Aikido training integrates several martial systems, including empty-hand (tai-jutsu), wooden sword (aiki-ken), wooden staff (aiki-jo) and knife (tanto).
Instead of clashing with an attacker’s physical strength, the self-defence techniques of Aikido blend with the attacker’s movements using the dynamic engagement strategies and spherical motion evolved from techniques of Japanese Swordsmanship, thus enabling both men and women to effectively defend themselves against multiple armed or unarmed attackers without the need to employ the destructive punching and kicking techniques that can often be associated with other combat arts.
Check out our main website Aikido Bayside to learn more
What is it?
The Secret Science of Combat Strategy presents an invaluable resource for anyone interested in martial arts hoping to improve their knowledge and skills
About the book
What are the best fighters feeling, thinking, and doing before, during, and after combat? The intriguing answer to this question seeks to satiate those who are curious about the secret world the martial artist inhabits.
Who is it for?
Whether you are a practitioner of any combat system or are simply a fan of mixed martial arts, this guide offers a peek into the heart and soul of elite fighters who are true martial artists.
About the Author
Jules Aib is the founder and chief instructor of World Aikido-Yoga (WAY). Aikido- Yoga seamlessly integrates the sophisticated martial-art of aikido with the ancient science of yoga.


The Holy Grail Layout using CSS Grid Layout. Read the blog post on bitsofco.de
Not working properly? You may need to enable it in your browser.
 Let’s see what Chef Dee got that they don’t want us to eat. I’m up to something. The key to more success is to have a lot of pillows. I told you all this before, when you have a swimming pool, do not use chlorine, use salt water, the healing, salt water is the healing. I’m up to something. Lion! We the best. To be successful you’ve got to work hard, to make history, simple, you’ve got to make it. Hammock talk come soon.
 Everyone has a choice. I pick my choice, squeaky clean. They don’t want us to eat. Let’s see what Chef Dee got that they don’t want us to eat. The weather is amazing, walk with me through the pathway of more success. Take this journey with me, Lion!


Salt is a new approach to infrastructure management. Easy enough to get running in minutes, scalable enough to manage tens of thousands of servers, and fast enough to communicate with them in seconds.
Ansible is a radically simple IT orchestration engine that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.


Fabflix
MAIN Employee
MetaData
New Star:
Star name:
(require)
Birth Year:
(optional)
New Movie:
Movie Title:
Year:
Director
Star name:
Genre:


--- layout: post title: DoCoMo In The “Lifestyle Infrastructure” Business date: 2006-04-20 00:00:00 categories: business mobile ---
DoCoMo In The “Lifestyle Infrastructure” Business: "Nakamura says the company is in the lifestyle infrastructure business — not selling connectivity, but developing services that allow mobile technology to pervade every aspect of its users’ lives. The focus isn’t so much on coming up with new mobile-specific applications and services, but rather ones that simplify and build on everyday life and experiences."


{{night.title}}
{{comment.body}}
Upvotes: {{comment.upvotes}}
Submit Your Comment
Comment
{{comment.body}}
Upvotes: {{comment.upvotes}}


nmap2html - files/1oa6t0sb9e0uahiln7ismrvlre1564946898.xml
nmap -p 21,22,80,23,55 -oX files/1oa6t0sb9e0uahiln7ismrvlre1564946898.xml theofficesuite.ie
1 hosts in file [1 up/0 down]
Started Sun Aug 4 19:28:18 2019, Finished Sun Aug 4 19:28:23 2019
-------
<IP_ADDRESS> "theofficesuite.ie" [ipv4]
tcp 80 open [syn-ack] http ()
-------


This material is based on this blog post: http://risnandar.wordpress.com/2013/09/08/how-to-install-and-setup-apache-lucene-solr-in-osx/
brew install solr
. This installed solr v.4.5.1cd /usr/local/Cellar/solr/4.5.1/libexec/example/
java -jar start.jar
cd /usr/local/Cellar/solr/4.5.1/libexec/example/exampledocs/
, then execute ./post.sh vidcard.xml
in that directory. This inserts some data into the example database. Try this query in your browser: http://localhost:8983/solr/select?q=:&wt=json
And we can now use the solr
R package to query the Solr database to get raw JSON data:
url <- 'http://localhost:8983/solr/select'
solr_search(q='*:*', base=url, raw=TRUE)
[1] "{\"responseHeader\":{\"status\":0,\"QTime\":1,\"params\":{\"start\":\"0\",
\"q\":\"*:*\",\"wt\":\"json\"}},\"response\":{\"numFound\":2,\"start\":0,\"docs\"
:[{\"id\":\"EN7800GTX/2DHTV/256M\",\"name\":\"ASUS Extreme N7800GTX/2DHTV (256 MB)\"
,\"manu\":\"ASUS Computer Inc.\",\"manu_id_s\":\"asus\",\"cat\":[\"electronics\",
\"graphics card\"],\"features\":[\"NVIDIA GeForce 7800 GTX GPU/VPU clocked at 486MHz\"
,\"256MB GDDR3 Memory clocked at 1.35GHz\",\"PCI Express x16\",\"Dual DVI connectors,
HDTV out, video input\",\"OpenGL 2.0, DirectX 9.0\"],\"weight\":16.0,\"price\":479.95,
\"price_c\":\"479.95,USD\",\"popularity\":7,\"store\":\"40.7143,-74.006\",\"inStock\"
:false,\"manufacturedate_dt\":\"2006-02-13T00:00:00Z\",\"_version_\":1455773551158099968},
{\"id\":\"100-435805\",\"name\":\"ATI Radeon X1900 XTX 512 MB PCIE Video Card\",\"manu\"
:\"ATI Technologies\",\"manu_id_s\":\"ati\",\"cat\":[\"electronics\",\"graphics card\"],
\"features\":[\"ATI RADEON X1900 GPU/VPU clocked at 650MHz\",\"512MB GDDR3 SDRAM
clocked at 1.55GHz\",\"PCI Express x16\",\"dual DVI, HDTV, svideo, composite out\",
\"OpenGL 2.0, DirectX 9.0\"],\"weight\":48.0,\"price\":649.99,\"price_c\":\"649.99,
USD\",\"popularity\":7,\"inStock\":false,\"manufacturedate_dt\":\"2006-02-13T00:00:00Z\",
\"store\":\"40.7143,-74.006\",\"_version_\":1455773551214723072}]}}\n"
attr(,"class")
[1] "sr_search"
attr(,"wt")
[1] "json"
Or parsed data to a data.frame (just looking at a few columns for brevity):
solr_search(q='*:*', base=url)[,c(1:3)]
id name manu
1 EN7800GTX/2DHTV/256M ASUS Extreme N7800GTX/2DHTV (256 MB) ASUS Computer Inc.
2 100-435805 ATI Radeon X1900 XTX 512 MB PCIE Video Card ATI Technologies
I'm a Mac user, so I'm not too familiar with Windows and Linux, but will get to this soon, or send a pull request with instructions in this file.


Next: Symbols as Forms, Previous: The Evaluation Model, Up: The Evaluation Model
Forms fall into three categories: symbols, conses, and self-evaluating objects. The following sections explain these categories.


jl777c
27-Sep-18 09:25 PM
no sense to make a big red flag, when anybody that understands the tech will understand this
due to relative scarcity, if PIRATE gets massively popular over next few months, it could easily go above 1 KMD
but over the next 8 months there will be a very large amount of PIRATE created
so unless there is some crazy viral growth, it seems a > 1 KMD price would be hard to sustain, but then again 1 KMD price is no magical ceiling. PIRATE is a totally independent chain and it will trade at whatever price the market deems it should trade at
2
2


+++draft = false date = 2009-11-11T18:07:00.0000000Z title = "Third day at TechEd" slug = "Third-day-at-TechEd" +++
Once again, I attended some very interesting talks at TechEd. This mornings sessions was entitled “The daily Scrum”, about doing Scrum and agile development. This was mostly a Q&A session with answers to many of the practical problems one might encounter when trying to be agile.
Next, I had a real hard time deciding between staying on the agile track and attending the “Tools and Agile Teams” talk versus hearing Don Syme speak about F#. I chose the F# session, which I think was a good choice. Don is one of the primary architects behind F#, so it would have been a shame not to hear him speak about it. This talk really drove home some points about F#, and why it helps you do parallel programming, with immutability, the async language construct and Agents. Another good point is that F# should not be used for anything, and in a large application, Don suggested that only a small DLL might be written in F# - it should be used as a tool, where you needed. Don also showed some really impressing demos, using Direct3D from F#.
After lunch, I attended Roy Osherove’s talk about unit testing. His main points where to write maintainable, consistent and readable unit tests, and proceeded to show this can be done. He suggested using test reviews in order to get started writing good unit tests, which I think is a very good idea. Very insightful talk.
The last session of today was about cloud computing: “ Deep Dive into Developing Line-of-Business Applications Running in the Cloud ”. I don’t think this was a good session. There was too much demoing of an app in the cloud, and too little talk about the actual architecture behind it. Also, the presenters neglected to do any introduction to the Azure tools, I guess they expected everyone attending to know about those in advance.


adv3Lite TutorialBy Eric Eve Table of Contents Introduction Copyright ©2012-2013 by Eric Eve All rights reserved. This book may be used and redistributed without charge subject to the conditions that it may not be altered, and no copyright notices or authorship attributions may be removed or modified. Release History adv3Lite version 0.3 November 2012 adv3Lite version 0.7 April 2013 |


directions_car
Parking Ticket Reimbursement
Registered Employees
Reimbursement
Job Sites
Employees:
Name
Title
Auto Information
add
Add Employee
arrow_drop_down
Select An Employee
Check-In
Check-Out
Location
Rate
Total Hours
Status
Click on the map to create a Job Site marker
(right click on the marker to edit or delete it)
Add Employee:
account_circle
User Name
User Title
directions_car
Make
Model
License Plate Number
Edit or delete this place
Job site name
Job site name
Job site name


This currency converter allows you to check how much you would get when you convert your currency into another currency. The currency converter shows you the true exchange rate, also known as the LIVE rate.


--- layout: archive title: Zulip Chat Archive permalink: /stream/113489-new-members/topic/ordered.20pairs.html ---
I'm defining a subset of the Cartesian product of two types, specifically:
def graph : set X × Y := {(x : X, f x) | x : X}
(f
is a function, x
has not been separately defined) What's the right notation for this (the ordered pairs, and also the set builder itself)?
what do you mean by "right notation"?
A notation that works in Lean.
do you mean set (X × Y)
?
oh
No, I mean for the ordered pairs.
universes u v variables {X : Type u} {Y : Type v} (f : X → Y) def graph : set (X × Y) := { z | z.2 = f z.1 }
Ah.
Thanks.
It's kind of annoying that you can't write {⟨x,y⟩ : X × Y | y = f x}
you can't define that in ZFC either
you need to use specification and an existential
I guess you could use set.image
or set.range
, whatever it's called
def graph' : set (X × Y) := set.range (λ x, ⟨x,f x⟩)
(after import data.set.basic
)
import data.set.lattice universes u v variables {X : Type u} {Y : Type v} (f : X → Y) def graph : set (X × Y) := ⨆ x : X, {(x, f x)}
What's ⨆
?
import data.set.basic variables (X Y : Type) (f : X → Y) def graph : set (X × Y) := {z : X × Y | z.2 = f z.1} def graph' : set (X × Y) := set.range (λ x, ⟨x,f x⟩) example : graph = graph' := sorry
set X
is a complete lattice
Abhi: ⨆
is notation so you can start with #print notation ⨆
but if you don't want to go down the lattice rabbithole you could try my homework :-)
I'd be able to do the homework myself if I knew how to prove A = B iff B = A
:-/
eq.symm
only goes one way
eq_comm
I'm not sure how to interpret the results of #print notation. All I get is '⨆':1024 binders ',':0 (scoped 0) := #0
What's a binder?
oh oops
yes that is impossible to interpret
A binder is something like forall or exists or lambda I think
Kenny is using ⨆
in the same sort of way that one would use those things.
Yeah, I can tell it's something like forall, but I'm not sure what the difference is.
I think you can click on the sup to go to definition
I get "no definition found"
It's the analogue of \bigcup
for lattices
It's in order/complete_lattice.lean
notation
⨆ binders
, r:(scoped f, supr f) := r
I taught Abhi about bigcup today in lectures :-)
Glad to see my lectures are coming in handy in his day to day life
"day to day life"
I don't understand that notation line at all. I think notation in Lean is somehow evil; most of it is fine and then there are some weird hacks which I don't get at all. The key point is supr
somehow, and then everything else is some notation mantra I guess
def supr [complete_lattice α] (s : ι → α) : α := Sup {a : α | ∃i : ι, a = s i}
so supr s
is the supremum of the range of s
Btw I'm trying def p1 (g : graph) : X := g.1
based on the earlier definition of graph
, but it doesn't work:
invalid field notation, type is not of the form (C ...) where C is a constant g has type ⁇
universe u variable {X : Type u} variable {Y : Type u} variable {f : X → Y} def graph : set (X × Y) := { g | g.2 = f g.1 } def p1 (g : graph) : X := g.1
I guess part of the problem is that G
is not a type -- I tried coercing it as ↑G
, but that just produces more errors.
I think the bigger problem is Lean doesn't know what you want to take the graph of
I can't make sense of that definition
I would make f
an explicit argument and then pass it in p1
what does g : graph
mean?
it's not a type
what does
g : graph
mean?
Yeah, I realise that, but I want to show (separately from the definition) it's a bijection from graph
to X
.
I would make
f
an explicit argument and then pass it inp1
Indeed that removes the first error. How, though?
Keep in mind that graph f
is not a type either, it's a set
lean knows to coerce from a set to a type, but maybe you want the subtype instead?
a set is a term not a Type Abhi. Did you come to my lecture today?
a set is a term not a Type Abhi. Did you come to my lecture today?
I realise that -- but how else would I later show it to be a bijection from graph to X?
If X
is a type and p : X -> Prop
then in Lean there are two very different ways to express what in set theory is just "the subset of X consisting of elements a
for which p a
is true"
it did took me some time to figure out that Kevin wants us to prove that the two big big functions graph
and graph'
are equal
so you should start with ext X Y f
One is the set {x : X | p x}
and one is the subtype {x : X // p x}
I am in the middle of showing my graph
is Kenny's supr
thing
I bet example : graph = graph' :=
isn't what Kevin wanted us to prove
I proved that using eq.comm
example : graph X Y f = graph' X Y f :=
import data.set.basic import data.set.lattice variables (X Y : Type) (f : X → Y) def graph : set (X × Y) := {z : X × Y | z.2 = f z.1} def graph' : set (X × Y) := set.range (λ x, ⟨x,f x⟩) def graph'' : set (X × Y) := ⨆ x : X, {(x, f x)} example : graph = graph' := sorry example : graph = graph'' := sorry
I did them both :-)
I am not proud of the second one though.
Re:the bijection, I can write def p1 (g : graph f) := g.1
(still defining graph
as a set because subtype gives me even more errors), but then p1
for some reason has the wrong type. Apparently g.1
isn't processed correctly (it still has type X × Y
and not X
).
that has to do with the fact that it is a subtype
g
is a pair of an element of X x Y
and a proof that this element satisfies z.2 = f z.1
Is the notation for ordered pairs different for subtypes?
so the X value is g.1.1
That works, but I don't get why.
(deleted)
or g.val.fst
Ok, I get the argument, but I'm not sure why it's defined that way (to include a proof).
If you didn't have to provide a proof when constructing a value of the subtype, then it would just be the same as the entire original type.
Why does the same convention not apply to sets?
I don't really understand the question. A set is not a type; perhaps that's the answer
{% endraw %}Last updated: Apr 24 2020 at 19:02 UTC


I ended the last post by highlighting differences in keywords of publications related to bioethics on the Web of Science that were used in consecutive years, in multiple but non-consectuive years, and in only a single year. In this post, I will predict the number of years in which a keyword is used to categorize journal articles.
Visualizations
Before employing statistical models, it would be prudent to examine visually the data. Here are the visualizations with which at ended the previous blog post.
consec.evo <- consec.evo %>%
mutate(usage.type = case_when(
consec == TRUE ~ "Persist",
consec == FALSE ~ "Recur",
is.na(consec) ~ "Single"
))
consec.evo %>%
ggplot(aes(x = year, y = n, color = usage.type)) +
geom_line() +
xlab("Year") +
ylab("Keyword count") +
ggtitle("Trends in keyword usage by usage type")
For this visualization, I renamed the consec
variable and its levels to usage.type
and persist, recur, and single. Persist indicates that a keyword has been used in multiple, consecutive years, and recur indicates that a keyword has been used in multiple, non-consecutive years. Single refers to keywords that have been employed in only one year. Over this roughly 27 year period, recurrent keywords categorize articles the most frequently and at an increasing rate. Persistent and single-use keywords occur at roughly the same rate, although there is a marked difference starting around 2013.
Now, let’s examine visually the distributions of the number of years for the persist and recur usage types.
keyword.evo %>%
filter(!is.na(consec)) %>% # NA = one year
ggplot(aes(num.yrs.active, color = consec, fill = consec)) +
geom_density(alpha = .5) +
scale_fill_discrete(name='Usage group',labels=c("Recur", "Persist"))+
scale_color_discrete(name='Usage group',labels=c("Recur", "Persist")) +
xlab("Number of years a keyword is active") +
ylab("Density") +
ggtitle("Distribution of years active by whether consecutive") +
theme_minimal()
The two distributions appear to differ greatly. Recur has more density spread farthest along the x axis, but persist has the highest maximum value. These two groups clearly exhibit distinct behaviors.
The models below will investigate these group differences in the number of years during which a keyword is active, but I first want to examine in further detail the patterns of usage among keywords that are employed during multiple years. The next chunk of code creates a new object called year.diffs
. To do so, I isolate the distinct years in which each unique keyword is used, take the difference between adjacent years in the vector (assigned to a variable called diffs
), and then note the position of each difference in the vector of differences (assigned to a variable called position
). I also include a keyword
variable to identify to which keyword the differences and positions refer. Each row is uniquely identified by keyword
plus position
, as diffs
might have multiple equivalent values for a given keyword.
year.diffs <- vector("list", length = nrow(test.bind))
for (i in 1:length(year.diffs)) {
year.diffs[[i]] <- test.bind %>%
filter(keyword == keyword.evo$keyword[i]) %>%
distinct(year) %>%
as.matrix() %>%
diff() %>%
as.data.frame() %>%
mutate(keyword = keyword.evo$keyword[i])
}
year.diffs <- bind_rows(year.diffs)
year.diffs <- year.diffs %>%
select(-.) %>%
rename(diffs = year) %>%
group_by(keyword) %>%
mutate(position = 1:n())
year.diffs <- year.diffs %>%
ungroup()
head(year.diffs)
## # A tibble: 6 x 3
## diffs keyword position
## <dbl> <chr> <int>
## 1 19.0 healing 1
## 2 2.00 healing 2
## 3 5.00 healing 3
## 4 4.00 culture 1
## 5 4.00 culture 2
## 6 1.00 culture 3
The head()
function at the end of this code provides an idea about the organization of year.diffs
. For example, one can see that healing appears three times in this object, which means that it originally was used during four years. There was a gap of 19 years between its first and second years used, 2 yeas between its second and third years used, and 5 between its third and fourth years used.
Given the large number of keywords that appear in multiple years with at least one non-consecutive jump, let’s check out the histogram of the maximum non-consecutive differences. The x-axis will begin at 2 because that is the smallest non-consecutive difference.
year.diffs %>%
filter(diffs > 1) %>%
group_by(keyword) %>%
summarise(diffs.max = max(diffs)) %>%
ggplot(aes(diffs.max)) +
geom_histogram() +
ggtitle("Density plot of max non-consecutive year differences") +
xlim(2, 30) +
xlab("Maximum non-consecutive difference in years used") +
ylab("Number of occurrences")
Now, I want to know, for each non-consecutive sequence, how many times are there non-consecutive years?
year.diffs %>%
group_by(keyword) %>%
filter(diffs > 1) %>%
summarise(num = n()) %>%
summarise(n_distinct(num)) # 9 distinct values
## # A tibble: 1 x 1
## `n_distinct(num)`
## <int>
## 1 9
year.diffs %>%
group_by(keyword) %>%
filter(diffs > 1) %>%
summarise(num = n()) %>%
ggplot(aes(as.factor(num))) +
geom_bar(show.legend = FALSE) +
xlab("Number of non-consecutive usages") +
ylab("Count") +
ggtitle("Frequency of unique values of non-consecutive uses")
This bar graph suggests that the plurality of nonconsecutively used keywords have only one nonconsecutive break, yet there is a considerable portion of other cases have multiple nonconsecutive breaks.
I also want to know when different sized breaks in continuity occur; the following graph will plot the difference position against the difference magnitude.
ggplot(year.diffs, aes(position, diffs)) +
geom_jitter(aes(color = as.factor(position)), show.legend = FALSE) +
xlab("Position") +
ylab("Difference") +
ggtitle("Plot of differences by position") +
theme_minimal()
This jitter plot presents interesting results. Similar to a boxplot, this graphic allows one to see the density of inter-year differences for each position. The minimum position is one (because a keyword must be used at least twice to have a difference).
This is a boxplot of the same data.
year.diffs %>%
ggplot(aes(position, diffs)) +
geom_boxplot(aes(group = as.factor(position)))
This boxplot does not look as nice as the jitter plot. More importantly, while it does indicate outliers, much of the information is incomprehensible due to the tight clustering of cases near zero.
Model estimation
Before I get to the model estimation, it is necessary to provide an overview of my analytic strategy. The basic unit in the models will be the keyword, which means the predictors will need to be attributes about the keyword, but what attributes—aside from the number of active years and usage group—are present in the data? Without getting into too much theoretical justification (this will be in the forthcoming paper), I am interested in how relational attributes of keywords qua product categories in the academic/scientific market influence their lifecycle.
One may derive networks among the keywords because each keyword occurence is associated with a unique journal article. Two approaches to arriving at this network present themselves. First, one may create a bipartite network in which keywords are connected to journal articles; second, one may employ the tidytext package to create bigrams from the author keyword cell. These approaches ultimately can result in the same data structure, and I will use both in this series on bioethics keywords. Two keywords will be connected if they were used to describe the same journal article, and the weight of their relationship will correspond to the number of papers in which they co-occured.
What are the attributes in which I am interested? The first two are degree and the local clustering coefficient: that is, the number of keywords to which each keyword is connected and how densely each of the keywords to which a focal keyword is connected are linked among themselves. Degree and clustering would seem to be positively related to the number of years in which a keyword is employed. If a keyword is related to many others, then it might be used more, which would increase the likelihood of its extended temporal distribution, and, if the keyword is a member of a densely connected group of keywords, then they might form a subspeciality associated with an active community of scholars. The remaining attributes pertain to the distribution of weights within the ego network (the set of relationships in which a focal keyword is involved) of each keyword.
The first things I will do is to determine which type of count model is best to estimate given the data; I here adjudicate between the two main contenders: Poisson and negative binomial regression models. For a good introduction to choosing between these two model types, check out UCLA’s Institute for Digital Research and Education.
Before I go any further, I want to remove keywords in the “single” usage group because it perfectly predicts a num.yrs.active
value of one. I also create a new variable, which is simply a factor version of consec
.
keyword.data.0 <- keyword.data.0 %>%
mutate(consec.factor = as.factor(consec))
keyword.data.reduced <- keyword.data.0 %>%
filter(consec.factor != "single")
The following summary indcates the variance is greater than the mean, which suggests that a negative bionomial model is more appropriate.
keyword.data.reduced %>%
filter(!is.na(num.yrs.active)) %>%
summarise(var(num.yrs.active), mean(num.yrs.active))
## # A tibble: 1 x 2
## `var(num.yrs.active)` `mean(num.yrs.active)`
## <dbl> <dbl>
## 1 23.9 5.54
However, we can explicitly test this by running both types of model and then using a likelihood ratio test for a comparison. I present the comparison between two base models containing only degree and local clustering coefficient, but the sustantive takeaway remains unchanged with the later models.
# negative binomial model
# MASS masks select from dplyr and tidygraph
nb.0 <- MASS::glm.nb(num.yrs.active ~ degree + lcc,
data = keyword.data.reduced,
control=glm.control(maxit=50))
# poisson model
pois.0 <- glm(num.yrs.active ~ degree + lcc,
family = "poisson",
data = keyword.data.reduced)
screenreg(list(pois.0, nb.0),
custom.model.names = c("Poisson", "Negative Binomial"))
##
## ===============================================
## Poisson Negative Binomial
## -----------------------------------------------
## (Intercept) 3.27 *** 3.12 ***
## (0.03) (0.04)
## degree 0.00 *** 0.01 ***
## (0.00) (0.00)
## lcc -3.59 *** -3.32 ***
## (0.07) (0.08)
## -----------------------------------------------
## AIC 9770.43 9599.46
## BIC 9787.54 9622.27
## Log Likelihood -4882.22 -4795.73
## Deviance 2420.53 1682.69
## Num. obs. 2215 2215
## ===============================================
## *** p < 0.001, ** p < 0.01, * p < 0.05
# which is preferable
chi2 <- 2 * (logLik(nb.0) - logLik(pois.0))
pchi2 <- pchisq(2 * (logLik(nb.0) - logLik(pois.0)), df = 1, lower.tail = FALSE)
print(paste0("Chi-squared value ", chi2))
## [1] "Chi-squared value<PHONE_NUMBER>40651"
print(paste0("P-value for chi-squared ", pchi2))
## [1] "P-value for chi-squared 1.65546825142021e-39"
As shown in the table, the log counts produced by the two models are pretty similar, with the local clustering coefficient having the largest discrepancy. The likelihood ratio test and its corresponding p-value indicate that the negative binomial model is preferable. I will employ negative binomial regression for the following models.
The next two models add attributes about tie weights (i.e., mean and range) as well as the consec
category.
nb.1 <- MASS::glm.nb(num.yrs.active ~ degree + lcc + mean + range,
data = keyword.data.reduced,
control=glm.control(maxit=100))
nb.2 <- MASS::glm.nb(num.yrs.active ~ degree + lcc + mean + range +
consec.factor,
data = keyword.data.reduced,
control=glm.control(maxit=100),
na.action = na.exclude)
screenreg(list(nb.0, nb.1, nb.2),
custom.coef.names = c("Intercept",
"Degree",
"Clustering",
"Mean (ego weight)",
"Range (ego weight)",
"Recur"))
##
## ============================================================
## Model 1 Model 2 Model 3
## ------------------------------------------------------------
## Intercept 3.12 *** 3.10 *** 2.86 ***
## (0.04) (0.04) (0.07)
## Degree 0.01 *** 0.01 *** 0.01 ***
## (0.00) (0.00) (0.00)
## Clustering -3.32 *** -3.33 *** -3.27 ***
## (0.08) (0.08) (0.08)
## Mean (ego weight) 0.02 *** 0.02 ***
## (0.00) (0.00)
## Range (ego weight) -0.01 *** -0.01 ***
## (0.00) (0.00)
## Recur 0.22 ***
## (0.06)
## ------------------------------------------------------------
## AIC 9599.46 9590.54 9576.68
## BIC 9622.27 9624.76 9616.60
## Log Likelihood -4795.73 -4789.27 -4781.34
## Deviance 1682.69 1680.90 1674.33
## Num. obs. 2215 2215 2215
## ============================================================
## *** p < 0.001, ** p < 0.01, * p < 0.05
The predictors added in Models 2 and 3 are significant at the p < .001 level, and the indicators at the bottom of the table suggest that there is a general immprovement. However, let’s run some tests to have further certainty.
nb.1.update <- update(nb.1, . ~ . - mean - range)
## test model differences with chi square test
anova(nb.1, nb.1.update) # two degree-of-freedom chi-square test
## Likelihood ratio tests of Negative Binomial Models
##
## Response: num.yrs.active
## Model theta Resid. df 2 x log-lik. Test
## 1 degree + lcc 17.80100 2212 -9591.456
## 2 degree + lcc + mean + range 18.22467 2210 -9578.543 1 vs 2
## df LR stat. Pr(Chi)
## 1
## 2 2 12.91285 0.001570399
This two degree-of-freedom chi-squared test indicates that Model 2 is preferable to Model 1. How does Model 2 compare to Model 3?
nb.2.update <- update(nb.2, . ~ . - consec.factor)
anova(nb.2, nb.2.update)
## Likelihood ratio tests of Negative Binomial Models
##
## Response: num.yrs.active
## Model theta Resid. df
## 1 degree + lcc + mean + range 18.22467 2210
## 2 degree + lcc + mean + range + consec.factor 18.59242 2209
## 2 x log-lik. Test df LR stat. Pr(Chi)
## 1 -9578.543
## 2 -9562.682 1 vs 2 1 15.8606 6.818372e-05
These results confirm the intuition from the table: Model 3 is the most preferable model.
Before interpreting Model 3, I will display again the results, and I will also include a coefficient plot from the texreg
package. A one unit increase in degree (i.e., a keyword having one extra connection to another keyword) corresponds to a miniscule .00676 increase in the log number of active years, and a one unit increase in the local clustering coefficient predicts a 3.27 decrease in a keyword’s log number of active years. The two attributes concerning the weights of ties in which a keyword is involved exhibit opposite effects. A one unit increase in the mean tie weight corresponds to a .0164 increase in the log number of active years, while, for a one unit increase in the range of a keyword’s ego network tie weights, the expected log number of active years decreases by .0143. Compared to keywords that appear in consecutive years (“persist”), non-consecutive words have a log number of active years that is higher by .220.
summary(nb.2)
##
## Call:
## MASS::glm.nb(formula = num.yrs.active ~ degree + lcc + mean +
## range + consec.factor, data = keyword.data.reduced, na.action = na.exclude,
## control = glm.control(maxit = 100), init.theta = 18.59241688,
## link = log)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -5.3945 -0.6687 -0.1368 0.4440 3.3291
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.8582343 0.0705716 40.501 < 2e-16 ***
## degree 0.0067637 0.0004916 13.758 < 2e-16 ***
## lcc -3.2703948 0.0790910 -41.350 < 2e-16 ***
## mean 0.0164384 0.0037479 4.386 1.15e-05 ***
## range -0.0143303 0.0036717 -3.903 9.50e-05 ***
## consec.factorrecur 0.2195409 0.0550104 3.991 6.58e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for Negative Binomial(18.5924) family taken to be 1)
##
## Null deviance: 5433.4 on 2214 degrees of freedom
## Residual deviance: 1674.3 on 2209 degrees of freedom
## AIC: 9576.7
##
## Number of Fisher Scoring iterations: 1
##
##
## Theta: 18.59
## Std. Err.: 1.80
##
## 2 x log-likelihood: -9562.682
plotreg(nb.2)
## Model 1: bars denote 0.5 (inner) resp. 0.95 (outer) confidence intervals (computed from standard errors).
Let’s also look at the results as incidence rates.
est <- cbind(Estimate = coef(nb.2), confint(nb.2))
exp(est)
## Estimate 2.5 % 97.5 %
## (Intercept) 17.43072183 15.10498804 20.08741103
## degree 1.00678661 1.00562108 1.00797485
## lcc 0.03799143 0.03245402 0.04443781
## mean 1.01657427 1.00750807 1.02581463
## range 0.98577188 0.97747124 0.99414979
## consec.factorrecur 1.24550485 1.11694638 1.39178571
Recurrent keywords have an incident rate that is 1.25 times persistent keywords, holding the other variables constant. The number of active years increases by almost 1% for a one unit increase in degree, and it decreases by roughly 96% for a one unit increase in the local clustering coefficient (which is drastic and the maximum possible increase). A one unit increase in the mean of the mean weight corresponds to an almost 2% increase in the incident rate of active years, while a one unit increase in weight range predicts a 1.4% decrease in the incident rate of num.yrs.active
.
The margins package provides a wonderful function (cplot
) for visualizing the change in estimates associated with different values of a predictor, holding constant the other predictors.
cplot(nb.2, "degree",
what = "prediction",
main = "Predicted # of active years, given degree",
xlab = "Degree")
In this first graph, one can see that the effect of degree on predicted value increases non-linearly, although the confidence intervals also greatly expand as the density of cases with very high degree is low.
cplot(nb.2, "lcc",
what = "prediction",
main = "Predicted # of active years, given clustering",
xlab = "Clutering coefficient")
Unlike degree, the local clustering coefficient has a non-linear inverse relationship with predicted number of years active.
cplot(nb.2, "mean",
what = "prediction",
main = "Predicted # of active years, given mean weight",
xlab = "Mean weight")
This plot echoes the results in the above tables. There is a very slight positive relationship between the mean of tie weights in a keyword’s ego network, yet the confidence intervals in the right half of the visualization prevents one from knowing whether or not the relationship is non-linear.
cplot(nb.2, "range",
what = "prediction",
main = "Predicted # of active years, given range of weights",
xlab = "Weight range")
Even though the confidence intervals are quite wide here, it is possible to identify a definite non-linear decrease in the relationship between the weight range and the predicted values. This trend
cplot(nb.2, "consec.factor",
what = "prediction",
main = "Predicted # of active years, given usage group",
xlab = "Usage group")
Finally, while the persist category has much wider confidence intervals, it is nonetheless clearly associated with lower estimate.
Finally, let’s look at a few last graphs. In this first visualization, I graphed the estimates against the observed number of active years for the keywords. For approximately the first (left-most) third of observed values, the predictions slightly overestimate, and, for the remaining observed values, the predictions underestimate num.yrs.active
. However, after 23 on the x-axis, many predictions are severe overestimates.
keyword.data.reduced <- keyword.data.reduced %>%
mutate(prediction = predict(nb.2, keyword.data.reduced,
type = "response"))
ggplot(keyword.data.reduced, aes(x = num.yrs.active,
y = prediction)) +
geom_point() +
geom_abline() +
ylab("Prediction") +
xlab("Observed umber of years active") +
ggtitle("Predicted values versus observed number of years active")
The next two visualizations compare, for the persist and recur usage groups, residuals against the observed and predicted number of years active values.
keyword.data.reduced <- keyword.data.reduced %>%
mutate(residual = num.yrs.active - prediction)
keyword.data.reduced %>%
filter(!is.na(consec.factor)) %>%
ggplot(aes(x = num.yrs.active,
y = residual,
color = consec.factor,
shape = consec.factor)) +
geom_point(show.legend = FALSE) +
geom_hline(yintercept = 0) +
ylim(-50, 50) +
facet_wrap(~ consec.factor, nrow = 1) +
xlab("Number of years active") +
ylab("Residual") +
ggtitle("Residual versus observed number of years active by usage group")
In the first of these two graphs, I have plotted the residuals versus observed values by group. Unfortunately, the seems to be a non-random relationship between the number of years active and the residuals.
Likewise, in the last graph where I visualize the relationship between the residuals and fitted values, there are obvious non-random relationships in both groups.
keyword.data.reduced %>%
filter(!is.na(consec.factor)) %>%
ggplot(aes(x = prediction,
y = residual,
color = consec.factor,
shape = consec.factor)) +
geom_point(show.legend = FALSE) +
geom_hline(yintercept = 0) +
ylim(-50, 50) +
xlim(0, 75) +
facet_wrap(~ consec.factor, nrow = 1) +
xlab("Predicted number of years active") +
ylab("Residual") +
ggtitle("Residual versus predicted number of years active by usage group")
What’s to come?
The visualizations and regression results in this post uncover interesting results. There are systematic differences between the keyword usage groups, particularly regarding the “lifespans” of these keywords, and the number of years in which keywords are active is associated—at least in part—with how each keyword is linked to the others. The negative binomial models presented here obviously pose problems in their predictive abilities. On the one hand, the incompleteness of the purely relational attributes should not be surprising. I included only four predictors, and other factors, such as attributes of the individuals who employ the categories, are surely at play. On the other hand, there is more to be done in specifying the relationships between the included predictors and num.yrs.active
, which will of course be the focus of the next post!


Intro Text
More Intro Text Link.
Layer 1 (Physical)
The physical entity e.g. heater, humidifier, e-bike, Baby Camera
Layer 2 (Sensors and Actuators)
In this layer, the physical entity has sensors, and actuating elements. For a baby camera, an example of the sensors are the optical sensor and motion sensor
Layer 3 (Control Layer)
In this layer, the sensors and actuators are connected to a microcontroller. E.g: Raspberry Pi, Arduino, Beaglebone Black
Layer 4 (Network Connectivity)
The microcontroller needs to transmit data over the network. E.g: here a baby camera transmits images and video through WiFi
Layer 5 (Data)
The data needs to be collected, analyzed and presented to the user. E.g: The user receives the images and videos from the baby camera on a mobile device


The upper part of the table follows the main organizing principles of the UD taxonomy such that rows correspond to functional categories in relation to the head (core arguments of clausal predicates, non-core dependents of clausal predicates, and dependents of nominals) while columns correspond to structural categories of the dependent
(nominals, clauses, modifier words, function words).
The lower part of the table lists relations that are not dependency relations in the narrow sense.
* The advmod relation is used for modifiers not only of predicates but also of other modifier words.
Individual languages may define more specific relations as subtypes of the universal types defined here.
A subtyped relation always starts with the basic type, followed by a colon and the subtype string.
In general, subtypes are language-specific and optional. However, some subtypes are assumed to apply
to many languages and they should be considered semi-mandatory: If the language has the phenomenon that
the subtype focuses on, then the subtype should be used. Examples of subtypes that should be used this
way include but are not limited to the following:


My name is Jose Brian Sandoval; however, I prefer Brian, my middle name. When given the opportunity, I tend to exceed expectations and complete every task to the best of my ability. I find patterns and problems; then deliver a reliable solution.
When I was 3 my family started a small business. They had a small shop that opened only on the weekends in Fletcher, North Carolina.
2 years later they decided to open a second business in Waynesville, NC. This Store would focus on providing cultural foods to our Hispanic community.
Our business became our life and as I grew older, our business grew. We expanded twice, and soon as I learned to count change, I begged to learn the cash register.
At 12 years old, I started counting the register. For each service we had, we adopted a separate system for doing it. By this point I started signing up for computer classes at school and learning Microsoft Word and Excel.
The older I got, the more I was eager to learn. By 14 started counting deposits and learning the end of the day reports. I began to make small changes where I thought we could improve. I also began learning the new systems first and then train the rest of the team.
By this point, our business had become my life. I came straight from school and helped my parents every day. At 16 years old, I was running the business and closing the store. My parents could now leave and have dinner alone or take a night off.
I became the person to implement changes across the business; whether a new system or services, I was the one to make it happen. This passion has drove me to pursue a degree in Business Administration.
As a Business major, gained the ability to pursue many different opportunities. Working at Marriott has provided me just that, an opportunity.
My education has provided me with a foundation of fundamental business principles, that I have seen first-hand through my work experience. Because of my drive and experience, my roles in the work place have always evolved and become more operations driven.
Working at a hotel for three years has opened my eyes to the industry. I've learned so much in a very short period. I started in July of 2016, during the final stages of a two-year renovation project.
As an Assistant Front Office Manager I've gained a tremendous amount of experience. I was fortunate to constantly be involved in bringing new systems and processes across the hotel. I gained a better understanding of how much goes into running a hotel, especially the hotel of the future.
Looking back, for as look as I can remember, I have always been the one to solve problems. In all of my roles I've always been the one my team looks to when something goes wrong. I am able to remain calm and collected as I find the best possible solution and minimize the risk.
This led me to pursue a career change that allowed me to use all of my skills to succeed. As a Project Coordinator, I am able to manage several tasks at hand while ensuring they continue to move forward. I enjoy a fast paced environments and understand how I fit into the bigger picture. I am responsive and provide communication of risks, before they become a problem. I've also become a resource for new team members to ask questions and learn from.
Currently I am a Project Manager, leading a small team of permit specialist through the permitting process. It's fast-paced and always changing, which pushes to be be more flexible and creative every single day. I strive to provide constant open communication to my clients with real time updates. I set realistic deadlines and constantly work to deliver faster results. I enjoy seeing everything from a broader perspective and being the lines of communication for multiple departments.
With my experience I've developed an ability to manage multiple projects concurrently and collaborate effectively with stakeholders. I possess exceptional problem-solving abilities with a detail-oriented approach.
Whether it's creating or implementing new technologies, I've develop and implement several new concepts and ideas in my previous roles. I thrive when giving the fundamentals tools needed and able to work freely through the motions. I quick find patterns and build processes for myself that will produce results effectively and efficiently.


2022-05-30
I've wanted to write about this for a long time. Not because I have anything special to say (or am even qualified to say it). I just get asked about this a lot and wanted something I could point to.
When most people think of design, they think of how something looks. Some may even say how it works. The phrase I've used since I joined Square is full stack design.
In software development, there is the concept of "full stack engineering" — building the entire system, considering the visible and invisible components. The same applies to design. Full stack design is the integration of brand, marketing, product, and research — everything from tone & voice to pixels & flows.
Few companies understand, exhibit, and nurture this mentality [1]. The ones who do are titans, with incredibly high brand loyalty and product NPS. So why is full stack design so important?
Good design is good business.
Duh.
That sounds great and all, but how should one promote full stack design within their company? I don't know [2]. But I can share a few indicators that may suggest a company embodies full stack design.
I am not the best person to speak about this topic. But I am someone who cares deeply about it. I saw the power of it first hand at Square and will apply it in every team I join.
I hope this post can put a name to something I believe is critical to building great products and great businesses. I hope it can motivate designers to think (even) more holistically. I hope it can encourage product managers (and other disciplines) to care more deeply about design. And I hope it inspires someone more qualified to write a better post.
Thank you to Arjun Mahanti for reading a draft of this post 🤍
[1] Good examples of companies who embody full stack design include Airbnb, Stripe, and Square (yes, I'm biased).
[2] Okay, I lied. I sort of know, or at least have some ideas. But that is a post for another day.
[3] As companies grow, there is often a pendulum swing between functional and divisional org structures. Functional structures centralize each discipline to optimize for mastery of craft and global decision making. Divisional structures organize around products or services with multiple disciplines rolling up to a single business unit leader (e.g. General Manager) who is responsible for P&L, optimizing for speed and local decision making. Neither is better. Each have a time and place. And you can mix and match them. For example, Square switched from functional to divisional with the creation of Cash App. But design was centralized for several more years. Even after design became siloed between business units, it remained centralized at different altitudes in the org chart — typically longer than other disciplines. This is how I knew design held a special (or at least, different) place in the org.
[4] I intentionally used the word "feeling". It is less about having a perfectly consistent visual design.  And that goes much deeper than whether the colors are the same (though that does matter).
[5] The difference is correlation vs causation. I said the list would suggest whether a company embodies full stack design, but it does not guarantee it.


This is a Java client adapter for Avalara AvaTax web services. This version
has support for the AvaTax account service. It is for internal use and
approved customers only.
The adapter was built using Apache Axis WSDL2Java to build the stubs which
were then enhanced to support a custom Profile SOAP header element and a WS-Security
UsernameToken SOAP header element.
A set of JUnit tests are provided as examples. You may wish to refer to .NET/COM
documentation for more details.
YOU MUST CONFIGURE avatax4j.properties before running tests or samples!
Use avatax4j.properties.sample as an example and insert your account and license.
| Contents |
| /doc/api |
Javadocs - API Documentation |
| /lib |
All required jars |
| /samples |
n/a
|
| /src |
Full source code |
| /test |
AccountSvcTest JUnit test class to be used as an example. |
| avatax4j.properties.sample |
Sample properties file for JUnit tests to configure url, account, license
This must be renamed to avatax4.properties and configured prior to running
the unit tests. |
| build.xml |
Ant build script. Run "ant" for usage instructions |
Dependencies
All dependent jar files are included for build and running.
-
JDK 5.0 or later
- avatax4j should be compatible with J2SDK
1.4.2 , but we no longer provide support for this version. You will
need to use the cacert file from JDK 5.0.
- Apache Ant
1.6.x or later
Getting Started
- Look at the examples in the samples directory.
- Look at the unit tests for more examples.
- Read the Javadocs
- Configure avatax4j.properties.Use avatax4j.properties.sample as an example
and insert your account and license.
- Run the samples and unit tests. See build instructions below.
Build Instructions
A complete Ant build.xml script is provided in the root directory. Here is
a list of the main target Ant tasks.
- compile - compiles everything
- javadoc - builds the javadocs
- test - runs the junit tests
- clean - cleans up all generated files and directories
- jar - creates the avataxAccount4j.jar file
- usage - provides help on using the build tool (default)
Deployment
Many applications will only need to deploy the avataxAccount4j.jar and avatax4j.properties
file. The avatax4j.properties file can be eliminated by setting the server url,
username and password credentials in your code. Examples of this can be found
in any of the junit test suites.
Some applications may require deployment of additional jar files from the lib
folder. These are all standard Sun, Apache, or other open-source components.
The version is indicated in most of the jar file names, but avatax4j should
be generally compatible with later versions. The axis-ant.jar, junit.jar, wsdl4j.jar
are only used for building and testing avatax4j.
Change History
<IP_ADDRESS> - Nov 2008
- Updated to the 5.3.0 AccountSvc interface.
4.15.0 - June 2007
- Eliminated backward compatibility issue with AvaTax server API changes.
Previously if new elements were added to the server api, it was necessary
to either get a new adapter or make sure that the avataxAccount4j.jar preceeded
axis.jar on the classpath. This was problematic or not possible in some environments.
We are handling this issue now and no longer depend on the classpath order.
Updating the adapter for server API changes is not necessary except when you
wish to use the new features. Backward compatibility will be maintained. We
highly recommend updgrading to this new adapter. No code changes should be
necessary. The interface has not been changed.
4.14.0 - May 2007
- Eliminated need for wss4j-1.1.0.jar, opensaml-1.0.1.jar, and xmlsec-1.2.1.jar
files. The SOAP security token is now directly built in BaseSvcSoapStub.setSecurity().
The only required code change is in the wsdd configuration. See the next change
for details.
- Eliminated the need for the avatax4j.wsdd file. You should simply call the
default AddressSvcLocator() or TaxSvcLocator() constructor instead of passing
a config file (new FileProvider("avatax4j.wsdd")). If you wish to
still use a wsdd config file, then you need to remove the <handler type="java:org.apache.ws.axis.security.WSDoAllSender"
> handler. This is because we no longer use wss4j. See any of the samples
or junit tests for code examples. The handler has been removed from the included
avataxj4.wsdd, but it isn't necessary if you just use the default locator
constructors.
- Included the activation.jar and mailapi.jar files to eliminate the annoying,
but inocuous "Unable to find required classes (javax.activation.DataHandler
and javax.mail.internet.MimeMultipart). Attachment support is disabled"
warning message.
Known Issues
- If you get a warning message regarding Xalan as follows:
Unable to patch xalan function table.
java.lang.NoSuchFieldException: m_functions
at java.lang.Class.getField(Unknown Source)
at org.apache.xml.security.Init.registerHereFunction(Init.java:429)
at org.apache.xml.security.Init.init(Init.java:124)… (and so on)
You need to copy the xalan-2.6.0.jar to your JAVA_HOME/lib/endorsed/ directory.
You may need to create the endorsed directory.
- Axis BeanDeserializer failes with unrecognized SOAP response elements.
This issue has been fixed in the 4.15.0 release
- Ant test target cannot find the junit task:
The easiest way to fix this is to copy the junit jar from the avatax4j/lib
directory to your ant/lib directory.
- Warning message: "Unable to find required classes (javax.activation.DataHandler
and javax.mail.internet.MimeMultipart). Attachment support is disabled"
occurs.
You need to include the activation.jar and mailapi.jar (included in releases
4.14.0 and later). Or you can download these at: http://java.sun.com/products/javabeans/jaf/downloads/index.html
and http://java.sun.com/products/javamail/downloads/index.html